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Abstract 


The  purpose  of  this  project  is  to  analyze  the  effects  of  Title  II  Weapon 
Systems  Acquisition  Reform  Act  (WSARA)  of  2009  on  the  Program  Executive  Office 
(PEO)  Combat  Support  and  Combat  Service  Support  (CS&CSS).  Our  methodology 
includes  a  literature  review  as  well  as  interviews  with  program  office  stakeholders. 
We  review  collected  data  to  establish  a  correlation  between  WSARA  implementation 
and  how  stakeholders  perceive  the  application  of  these  regulations  through  the 
following: 

•  Identify  the  intended  effect  of  Title  II  WSARA  regulations  on  the 
acquisition  process  concerning  cost,  schedule,  and  performance. 

•  Determine  the  extent  to  which  enacted  Title  II  WSARA  has  been 
applied  within  PEO  CS&CSS  programs. 

•  Identify  gaps  between  the  intent  and  application  of  Title  II  WSARA 
within  PEO  CS&CSS,  offering  recommendations  for  further  research. 

The  analysis  shows  that  correlations  exist  between  WSARA  implementation 
and  improved  cost  and  schedule  performance. 

Keywords:  Weapon  Systems  Acquisition  Reform  Act  (WSARA),  Program 
Executive  Office  (PEO) 
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I.  INTRODUCTION 


A.  ACQUISITION  PROCESS  AND  THE  PERCEPTIONS  OF  REFORM 

Defense  spending  currently  accounts  for  over  13%  of  U.S.  government 
outlays,  at  an  anticipated  dollar  amount  of  $800  billion  for  fiscal  year  (FY)  2014. 
Acquisitions,  or  procurement  (these  terms  are  used  interchangeably),  spending  is 
approximately  17.5%  of  defense  outlays,  or  nearly  $140  billion  annually  (projected 
for  FY2014).  These  numbers  give  an  order  of  magnitude  to  the  funds  committed 
annually  to  create  capabilities  within  the  U.S.  Department  of  Defense  (DoD; 
Government  Spending  Breakdown,  n.d.). 

Concern  over  the  acquisition  process  appears  warranted  when  considering 
the  amount  of  taxpayer  dollars  funding  this  process.  The  Weapon  Systems 
Acquisition  Reform  Act  (WSARA)  of  2009  is  one  of  many  regulatory  reforms  aimed 
at  improving  the  complex  acquisition  process.  The  following  sections  present  a  brief 
background  on  the  acquisition  process  (system)  and  historical  efforts  at  reforming 
this  process  leading  up  to  the  implementation  of  the  WSARA  in  2009. 

B.  THE  ACQUISITION  SYSTEM  “BIG  A” 

The  acquisition  system  is  composed  of  the  requirements  identification 
process,  the  budgeting  and  funding  process,  and  the  actual  process  of  managing  the 
purchase  or  manufacture  of  the  product.  In  defense  terminology,  these  three  areas 
are  as  follows: 

•  The  Joint  Capabilities  Integration  and  Deveiopment  System 
(JCIDS):  This  system  is  responsible  for  determining  requirements  and 
reducing  program  redundancy  across  military  service  branches. 

•  The  Planning,  Programming,  Budgeting,  and  Execution  System 
(PPBES):  This  system  is  responsible  for  forecasting  funding  resources 
and  managing  the  execution  of  congressionally  allocated  and 
appropriated  funds. 

•  The  defense  acquisition  system  (DAS):  This  is  the  actual  process  of 
incorporating  user  requirements,  determining  whether  to  make  or  buy  a 
product  to  meet  those  requirements,  and  the  method  spanning  from 
this  requirement  identification  through  the  entire  life  cycle  of  the 
resulting  product.  This  is  also  the  focal  point  for  acquisition  reform 
efforts  (Schwartz,  2013). 

When  meshed  together,  these  three  areas  contribute  to  the  entire  acquisition 
process  (“Big  A”),  as  depicted  in  Figure  1 . 
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Figure  1.  The  Acquisition  System 

(Schwartz,  2013) 

C.  THE  PLANNING,  PROGRAMMING,  BUDGETING,  AND  EXECUTION 
SYSTEM 

“A  number  of  issues  related  to  planning  and  budgeting  for  national  defense 
confound  DOD  and  congressional  decision  makers  annually.  Among  these  are  how 
to  perform  effective  and  competent  threat  assessment  and  the  consequences  of 
doing  this  job  well  or  poorly”  (Jones  &  McCaffery,  2008,  p.  88).  If  there  were  no 
need  to  consider  fiscal  constraints  when  outfitting  U.S.  military  forces,  there  would 
be  no  need  for  the  PPBES.  However,  in  an  environment  of  constraints,  the  PPBES 
is  the  U.S.  government’s  method  of  accounting  for,  planning,  programming,  and 
budgeting  for  products  and  systems  to  meet  the  needs  of  warfighters  and  monitor 
the  execution  of  funds  to  ensure  appropriate  fund  allocation  and  expenditures.  The 
PPBES  is  composed  of  four  steps: 

1 .  Planning:  During  the  planning  stage,  the  needs  of  combatant 
commands  are  analyzed  and  the  findings  are  published  in  the  Joint 
Programming  Guidance  document,  which  guides  the  DoD  components’ 
efforts  to  propose  acquisition  programs. 

2.  Programming:  During  the  programming  stage,  proposed  programs 
are  fleshed  out,  and  the  program  objective  memorandum,  a  document 
that  outlines  the  anticipated  missions  and  objectives  of  the  proposed 
weapon  system  and  anticipated  budget  requirements,  is  submitted  to 
propose  these  programs.  The  memoranda  are  reviewed  and,  as 
deemed  appropriate,  integrated  into  an  overall  defense  program. 
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3.  Budgeting:  Budgeting  occurs  concurrently  with  the  programming 
stage.  Proposed  budgets  are  reviewed  in  a  different  manner  than 
proposed  programs.  Upon  completion  of  a  program  decision  or  as  a 
result  of  a  budget  review,  program  budget  decisions  are  issued. 

4.  Execution:  Execution  occurs  simultaneously  with  the  program  and 
budget  reviews.  During  execution,  programs  are  evaluated  and 
measured  against  established  performance  metrics,  including  the  rates 
of  funding  obligation  and  expenditures.  (Schwartz,  2013) 

Much  of  acquisition  reform  results  from  discrepancies  in  what  occurred  during 
the  planning  and  programming  phases  when  compared  to  actual  budget  execution 
within  acquisition  programs. 

D.  THE  ACQUISITION  PROCESS  “LITTLE  A” 

The  U.S.  Department  of  Defense  (DoD)  is  by  far  the  largest  and  most 
complex  business  organization  in  the  world.  It  operates  more  than  5,400 
installations  worldwide  and  executes  more  than  1 5  million  contracts  per  year.  It  also 
develops  and  produces  the  most  sought-after  weapons  and  equipment  in  the  free 
world  (Fox,  201 1 ,  p.  1). 

The  DoD  is  complex,  and  the  acquisition  process  is  one  of  the  most  complex 
processes  existing  in  the  DoD.  Figure  2  portrays  the  acquisition  process,  its  phases, 
and  milestones.  Although  the  figure  seems  simple,  the  workings  of  the  process  are 
complex. 


Figure  2.  Acquisition  Process 

(Defense  Acquisition  University  [DAU],  n.d.-a) 

The  Material  Development  Decision  initiates  the  Material  Solution  Analysis 
(MSA)  phase  of  the  acquisition  process  after  a  user  need  is  identified.  The  goal  of 
this  phase  is  to  determine  the  method  (make  or  buy)  required  to  meet  end-user 
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needs.  This  phase  includes  the  Analysis  of  Alternatives,  where  acquisition  officials 
consider  any  capability,  including  commercial  off-the-shelf,  which  can  satisfy  the 
requirement  with  the  greatest  benefit  regarding  cost,  schedule,  and  performance. 
Emphasis  during  this  phase  is  on  adequately  identifying  user  needs  so  that  there  are 
no  requirement  misunderstandings  in  the  early  stages  of  program  consideration. 

The  program  moves  to  Milestone  A  with  the  creation  of  the  Technology 
Development  Strategy,  as  it  postures  for  progression  into  the  Technology 
Development  (TD)  phase.  The  Milestone  Decision  Authority  (MDA)  evaluates  the 
program  for  advancement  into  the  TD  phase  (DoD,  2013b). 

The  goal  of  the  TD  phase  is  to  produce  technologies  mature  and  capable 
enough  to  support  program  requirements.  The  acquisition  strategy — the  framework 
for  the  entire  program  acquisition  process — is  developed  during  this  phase  and 
includes  developing  the  system’s  cost,  schedule,  and  performance  criteria.  Product 
prototypes  are  developed  to  demonstrate  initial  capabilities  required  to  move  into  the 
following  acquisition  process  phase.  Requests  for  proposals  are  disseminated,  and 
contractors  begin  to  compete  for  the  contract  award.  The  program  manager  (PM) 
develops  the  preliminary  design  review,  which  establishes  baselines  for  human, 
software,  and  hardware  support  systems.  At  Milestone  B,  the  MDA  determines 
whether  to  allow  a  program  to  advance  into  the  Engineering  and  Manufacturing 
Development  (EMD)  phase  based  on  technological  capabilities  demonstrated  and 
technological  risk  reduced  during  the  TD  phase  (DoD,  2013b). 

A  program  advances  into  the  EMD  phase  if  the  technology  has  matured,  the 
MDA  has  approved  the  acquisition  strategy,  and  funding  is  fully  approved.  The 
intent  of  the  EMD  phase  is  to  demonstrate  sound  manufacturing,  logistical,  and  end- 
product  capabilities.  Collaboration  with  end  users  during  the  prior  two  acquisition 
phases  proves  imperative  because  in  those  phases,  key  performance  parameters 
(KPPs)  are  identified  as  the  baseline  by  which  the  product  will  be  analyzed. 
Significant  testing  occurs  during  this  phase,  and  products  must  demonstrate  their 
capabilities  in  a  relevant  developmental  environment.  The  EMD  phase  incorporates 
all  manufacturing,  industrial,  logistical,  and  human  systems  into  the  program,  and 
these  systems  must  demonstrate  their  capability  prior  to  the  next  milestone  decision. 
When  the  program  has  demonstrated  these  capabilities  and  successfully  completed 
developmental  test  and  evaluation  (DT&E),  the  MDA  conducts  the  Milestone  C  and 
low-rate  initial  production  (LRIP)  analysis,  which  determines  whether  the  program 
advances  to  the  Production  and  Deployment  phase  (DoD,  2013b). 

The  DOD  5000.02  states  that  “the  purpose  of  the  Production  and  Deployment 
phase  is  to  achieve  an  operational  capability  that  satisfies  mission  needs”  (DoD, 
2013b,  p.  27).  Operational  Test  and  Evaluation  tests  the  product  or  system  in  an 
operational  environment  (OE),  incorporating  end  users  and  support  systems 
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equivalent  to  the  OE.  After  the  system  demonstrates  capacity  to  meet  operational 
needs  and  meets  other  Milestone  C  criteria  mentioned  in  DoD  Instruction  (DoDI) 
5000.02  Enclosure  9,  the  LRIP  decision  allows  initial  production  to  begin.  Key  to  this 
phase  is  the  rectification  of  any  discrepancies  identified  during  the  previous  phase, 
which  ensures  that  issues  identified  during  test  and  evaluation  do  not  persist  in  the 
LRIP.  Furthermore,  as  manufacturing,  logistical,  and  industrial  systems  prove  their 
capability  to  produce  the  initial  low  rates  of  production,  these  systems  are  further 
assessed  to  determine  their  capacity  to  move  into  full-rate  production  (FRP).  FRP 
leads  into  the  full  deployment  of  the  product  or  system  to  the  intended  end  user 
(DoD,  2013b). 

The  final  phase  of  the  acquisition  process  is  the  Operations  and  Support  (OS) 
phase.  Prior  to  entering  the  OS  phase,  a  program  must  demonstrate  that  it  has  a 
life-cycle  sustainment  plan  (LCSP),  among  other  items.  The  LCSP  is  part  of  the 
program  from  the  beginning  and  develops  in  conjunction  with  the  maturing  program, 
ensuring  that  support  responsibilities  are  adequately  defined  and  that  trained 
personnel,  equipment,  and  infrastructure  fulfilling  support  and  maintenance  roles  are 
present.  This  phase  spans  the  remaining  life  cycle  of  the  program,  from  FRP  to  the 
end  of  the  product  or  system’s  life.  The  program  office  continues  to  monitor  the 
product  or  system’s  performance  data,  determining  whether  adjustments  must  be 
made  to  units  in  production  (DoD,  2013b). 

E.  THE  JOINT  READINESS  OVERSIGHT  COUNCIL  AND  JCIDS 

What  drives  a  requirement?  Jones  and  McCaffery  (2008)  may  have  stated  it 
best  when  they  said,  “Threats  to  national  security,  perceived  or  actual,  and  political 
priorities  drive  the  amount  of  defense  funding  requested  and  appropriated  for 
weapons  acquisition”  (p.  83).  The  Chairman  of  the  Joint  Chiefs  of  Staff  (CJCS) 
leads  the  Joint  Readiness  Oversight  Council  (JROC)  through  the  JCIDS  process 
and  the  JROC’s  integration  into  the  acquisition  process.  It  is  imperative  that  the 
JROC  be  involved  with  the  acquisition  process  from  the  moment  that  an  end-user 
requirement  is  identified.  This  involvement  ensures  that  programs  do  not  suffer  from 
redundancies  if  a  capability  exists  in  a  separate  military  branch,  allowing  for  more 
efficient  allocation  of  procurement  funds.  “JCIDS  provides  a  transparent  process 
that  allows  the  JROC  to  balance  joint  equities  and  make  informed  decisions  on 
validation  and  prioritization  of  capability  requirements”  (DAU,  n.d.-b,  p.  1).  The 
JROC  and  JCIDS  fulfill  many  key  roles  within  the  acquisition  process,  as  highlighted 
in  the  purple  portions  of  Figure  3. 
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Figure  3.  JCIDS  Review  Process 

(DoD,  2013b) 


DoD  components  and  services  conduct  capabilities-based  assessments  that 
lead  to  a  determination  of  whether  a  capability  gap  exists.  JCIDS  plays  a  key  role  by 
increasing  collaboration  among  services.  When  a  capability  gap  is  determined,  an 
initial  capabilities  document  (ICD)  is  generated  if  non-material  solutions  have  been 
exhausted.  Non-material  solutions  are  analyzed  on  the  basis  of  doctrine, 
organization,  training,  materiel,  leadership,  personnel,  facilities,  and  policy 
(DOTMLPF-P).  The  JROC  assists  with  determining  whether  the  requirement  can  be 
met  by  changing  an  aspect  of  DOTMLPF-P;  otherwise,  an  ICD  is  developed.  The 
ICD  is  the  first  key  document  that  JCIDS  contributes  to  the  acquisition  system.  This 
document  feeds  into  the  MSA  and  the  Concept  Exploration  phase  (CJCS,  2012). 

JCIDS  and  JROC  involvement  in  the  acquisition  process  remains  imperative 
to  meeting  user  needs.  JCIDS  develops  capability  development  documents,  which 
verify  that  the  developing  product,  system,  and  technology  are  capable  of  moving 
into  the  next  phase  of  the  acquisition  process.  The  JROC  regularly  interfaces  with 
acquisition’s  representatives  and  combatant  units,  verifying  KPPs  and  ensuring  that 
the  program  is  on  track  with  the  needs  of  joint  forces  and  cost,  schedule,  and 
performance  parameters.  JCIDS  is  also  responsible  for  developing  the  capabilities 
production  documents,  which  indicate  that  the  program’s  producability  and  logistical 
and  infrastructural  support  systems  are  mature  and  capable  enough  for  the  program 
to  exit  one  phase  and  enter  the  next  (DoD,  2013b). 

This  complicated  process,  composed  of  multiple  organizations  and  many 
stakeholders,  has  seen  many  reformation  initiatives  and  regulations  over  the  years. 
Intended  benefits  of  reform  varied  from  a  more  streamlined  process,  higher  end-user 
satisfaction,  and  control  of  cost  and  schedule  overruns.  Prior  to  understanding  why 
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reform  has  occurred  and  whether  reform  actions  have  been  successful,  one  must 
understand  the  process,  its  complexities,  and  the  differing  incentives  driving 
stakeholders  in  and  around  the  acquisition  process. 

F.  PERCEPTIONS  REGARDING  ACQUISITION  IMPLEMENTATION 

Acquisition  reform  dates  back  to  as  early  as  1 862  when  President  Abraham 
Lincoln  requested  the  resignation  of  the  secretary  of  war,  Simon  Cameron,  due  to 
contracting  “corruption  and  mismanagment”  (Schwartz,  2013,  p.  12).  Other 
acquisition  reforms  continued  in  the  decades  that  followed,  including  in  1958,  when 
the  Defense  Reorganization  Act  further  restructured  the  acquisition  process. 

Modern  acquisition  reform  initiatives  date  back  to  the  early  1990s  and  encompass  a 
list  of  63  initiatives  (Hanks,  Axelband,  Lindsay,  Malik,  &  Steele,  2005).  More  recent 
efforts  led  to  overhauls  of  major  defense  acquisition  policy,  including  the  acquisition 
operations  manual  DoDI  5000  series.  However,  despite  many  efforts  to  reform  the 
acquisition  system,  multiple  programs  continued  to  fail  at  meeting  cost  and  schedule 
requirements  (see  Figure  4). 
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Fiscal  Year 

2003 

2007 

2006 

Portfolio  size 

Number  of  programs 

77 

95 

96 

Total  planned  commitments 

$1.2  trillion 

$1.6  trillion 

$1 .6  trillion 

Commitments  outstanding 

$724.2  billion 

S875.2  billion 

S786.3  billion 

Portfolio  indicators 

Change  to  total  RDT&E  costs  from  first  estimate 

37  percent 

40  percent 

42  percent 

Change  to  total  acquisition  cost  from  first 
estimate 

19  percent 

26  percent 

25  percent 

Total  acquisition  cost  growth 

$183  billion 

$301.3  billion* 

$296.4  billion 

Share  of  programs  with  25  percent  increase  in 
program  acquisition  unit  cost  growth 

41  percent 

44  percent 

42  percent 

Average  schedule  delay  in  delivering  initial 
capabilities 

18  months 

21  months 

22  months 

Source:  GAO  analysis  of  DoD  data. 


Notes;  Data  was  obtained  from  DoD’s  Selected  Acquisition  Reports  (SAR)  (dated  December 
2002,  2006,  and  2007).  In  a  few  cases,  data  were  obtained  directly  from  program  offices.  The 
number  of  programs  reflects  the  programs  with  SARs;  however,  in  our  analysis  we  have  broken 
a  few  SAR  programs  into  smaller  elements  or  programs.  Not  all  programs  had  comparable  cost 
and  schedule  data  and  these  programs  were  excluded  from  the  analysis  where  appropriate. 
Portfolio  performance  data  do  not  include  costs  of  developing  Missile  Defense  Agency 
elements  or  the  Defense  Integrated  Military  Human  Resources  System  (DIMHRS)  program. 
The  total  acquisition  cost  growth  for  the  2007  portfolio  was  $295  billion  in  2008  constant  dollars 
(GAO-09-663T,  p.  0). 

Figure  4.  Analysis  of  DoD  MDAP  Portfolios 

(Francis,  2009) 

These  failures  to  meet  requirements  led  to  another  round  of  acquisition 
reforms,  including  the  WSARA  of  2009  and  the  Better  Buying  Power  initiatives. 

These  reforms  again  intended  to  reduce  cost  and  schedule  overruns  while  improving 
performance. 

G.  CONCLUSION 

The  number  of  initiatives  spanning  approximately  23  years  of  acquisitions — 
63  initiatives  in  total — is  one  of  the  key  points  from  the  RAND  study  conducted  by 
Hanks  et  al.  (2005).  These  initiatives  are  often  mere  policy  documents  generated  as 
a  new  leader  assumes  position  within  the  purview  of  the  acquisition  process  and 
rarely  find  their  way  into  actual  acquisition  regulation.  Another  issue  plaguing 
acquisitions  and  reform  efforts  is  discussed  in  a  more  recent  study  conducted  by  the 
Government  Accountability  Office  (GAO;  Francis,  2013).  This  study  concluded  that 
the  incentives  driving  multiple  stakeholders  involved  with  the  acquisition  process 
differ  vastly  and  continue  to  frustrate  the  acquisition  system.  Our  research  analyzes 
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incentives  and  how  current  reward  and  evaluation  systems  may  not  be  aligned  with 
regulation  implementations  (such  as  the  WSARA  of  2009)  and  create  a  disconnect 
between  regulation  and  activities  in  an  acquisition  organization. 
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II.  LITERATURE  REVIEW  AND  WSARA  BACKGROUND 


In  Chapter  I,  we  established  the  foundation  of  this  research.  In  this  chapter, 
we  provide  a  literature  review  surrounding  the  WSARA.  This  review  covers  the 
following: 

•  establishment  and  implementation  of  the  WSARA; 

•  WSARA  implementation  and  program  and  product  successes; 

•  desired  effects  occurring  from  the  WSARA;  and 

•  cost,  schedule,  and  performance  and  contract  metrics. 

To  capture  the  complex  timeline  of  reform  initiatives,  the  paragraphs  in  this 
chapter  are  organized  by  the  project’s  objectives;  resources  are  described  in 
chronological  order. 

A.  LITERATURE  SELECTION 

The  literature  surrounding  acquisition  reform  helps  to  illuminate  and 
determine  the  project  objectives.  To  determine  whether  there  is  a  correlation 
between  WSARA  implementation  and  program  and  product  success,  we  first 
examine  the  need  for  establishment  of  the  WSARA  and  the  key  aspects  of 
acquisition  reform.  Next,  to  determine  whether  WSARA  legislation  is  having  the 
desired  effects,  we  address  current  legislative  debate  and  articles.  Finally,  to  assess 
the  effects  of  WSARA  legislation  on  cost,  schedule,  performance,  and  contract 
metrics,  we  look  at  the  WSARA’s  intended  effects,  perceived  effects,  and  actual 
effects.  We  also  show  how  effects-based  management  theory  can  be  tied  to 
WSARA  implementation  and  describe  why  there  is  a  need  for  an  accurate 
assessment  on  the  measures  of  performance  related  to  the  WSARA. 

We  selected  the  literature  because  there  is  not  much  data  on  the  WSARA  or 
its  actual  effects.  We  chose  a  variety  of  mediums  due  to  the  ambiguity  surrounding 
the  topic.  To  see  more  deeply  into  the  subject  matter,  we  selected  journal  articles 
and  books  from  experts  on  acquisition  reform.  We  also  selected  key  GAO  reports  to 
illustrate  ongoing  challenges  and  analysis.  We  also  chose  congressional  transcripts 
that  cover  acquisition  reform  and  some  debate  on  the  WSARA.  And  finally,  we 
selected  relevant  news  articles  to  reflect  public  understanding  and  interpretation  of 
the  legislation  and  acquisition  reform  issues. 

1 .  Establishing  a  Need  for  the  WSARA 

The  WSARA  was  a  necessary  piece  of  legislation.  Its  intent  was  “to  improve 
the  organization  and  procedures  of  the  Department  of  Defense  for  the  acquisition  of 
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major  weapon  systems,  and  for  other  purposes”  (Weapon  Systems  Acquisition 
Reform  Act  [WSARA],  2009).  Decades  of  acquisition  reform  and  initiatives  led  up  to 
the  implementation  of  the  law.  Hanks  et  al.  (2005)  identified  63  acquisition  reform 
initiatives  from  the  1960s  to  2001  that  were  aimed  at  streamlining  logistics; 
improving  the  industrial  base;  managing  total  life-cycle  system  management;  and 
reducing  waste,  fraud,  and  abuse. 

The  Military  Reform  Caucus  (MRC),  cofounded  in  part  by  Newt  Gingrich,  was 
originally  founded  in  1981  (Malishenko,  1987).  Malishenko  (1987)  highlighted 
congressional  dynamics  in  his  article  with  the  MRC.  The  article  sheds  light  on  early 
attempts  for  bipartisanship  between  political  parties  of  the  House  and  Senate  in  the 
1980s  during  the  early  years  of  the  acquisition  reform  initiatives.  The  MRC 
membership  was  based  on  interested  congressional  members  and  helped  lay  the 
foundation  for  future  legislation  (Malishenko,  1987). 

Thompson  (1992)  described  his  frustration  with  the  number  of  laws  and  extra 
layers  of  bureaucracy  added  to  the  DoD  acquisition  processes  and  stated  that 
“greatly  increased  regulation  and  oversight  [are]  needed  to  fix  the  system”  (p.  729). 
Furthermore,  Congress  “added  an  assortment  of  new  laws  aimed  at  further 
disciplining  DOD  acquisitions  to  the  existing  1 ,150  feet  of  legislation  and  case  law 
governing  procurement”  (Thompson,  1992,  p.  729).  Thompson  (1992)  highlighted 
that 

most  acquisition  experts  call  for  the  elimination  of  the  statutes  that  limit 
the  proper  exercise  of  the  acquisition  officer’s  discretion.  They  all  call 
for  a  substantial  reduction  in  the  size  and  scope  of  the  systems 
commands,  systems  specifications,  evaluation  teams,  and  in  the 
number  of  criteria  the  evaluation  teams  consider  in  the  source 
selection  process.  Most  call  for  clear  command  channels,  limited 
reporting  requirements,  and  project  teams  with  small  high-quality  staffs, 
(p.  744) 

Many  reform  initiatives  were  established  leading  up  to  the  WSARA  law.  Most 
notable  is  President  Reagan’s  Packard  Commission  report  from  1986.  The  Packard 
Commission  “concluded  that  the  primary  problems  with  the  acquisition  process  were 
the  same  ones  identified  in  previous  decades  (cost  growth,  schedule  delays, 
performance  shortfalls)”  (Christensen,  Stearle,  &  Vickery,  1999,  p.  251).  Through 
their  study,  Christensen  et  al.  (1999)  highlighted  previous  attempts  of  legislation  that 
were  not  adequate  in  improving  cost  performances:  “Despite  the  implementation  of 
more  than  two  dozen  regulatory  and  administration  initiatives,  there  has  been  no 
substantial  improvement  in  the  cost  performance  of  defense  programs  for  more  than 
30  years”  (p.  252). 

A  2005  RAND  report  (Hanks  et  al.,  2005)  highlighted  acquisition  reform 
initiatives  that  illustrate  63  initiatives  from  1966  up  to  2001  that  were  marginally 
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successful.  Several  of  the  interview  quotations  from  the  RAND  report  emphasized 
this  theme: 

•  “Acquisition  Reform  inhibitors:  need  to  change  law  or  just  working  the 

edges;  need  to  change  how  funding  is  done.  Example: _ program 

subsidizes  everything  at _ (Major  Subordinate  Command),  so  the 

incentives  are  to  keep  selling  parts  [to  that  program]  and  not  let  them 
fix  the  parts”  (program  executive  officer-military;  p.  121); 

•  “The  test  community  is  still  focused  on  their  reporting  requirements 
rather  than  testing  to  fix”  (PM-military;  p.  110). 

In  their  analysis  of  organizational  transformation,  Eide  and  Allen  (2012) 
explained  acquisition  reform  legislation  in  clear  and  concise  detail.  Starting  with  the 
Packard  Commission  report  of  1986,  Eide  and  Allen  (2012)  specifically  stated. 

Excellence  in  defense  management  will  not  and  cannot  emerge  by 
legislation  or  directive.  Excellence  requires  the  opposite — 
responsibility  and  authority  placed  firmly  in  the  hands  of  those  at  the 
working  level,  who  have  knowledge  and  enthusiasm  for  the  tasks  at 
hand.  (p.  102) 

The  Goldwater-Nichols  Act  of  1986  followed  the  Packard  Commission  report 
and  addressed  some  of  the  changes  that  the  Packard  Commission  suggested, 
specifically  “diluted  authority  for  execution”  (Eide  &  Allen,  201 2,  p.  1 02).  Moving  up 
a  decade  into  the  1990s,  a  more  business-minded  aspect  became  apparent  in 
legislation  and  reform  initiatives.  Focusing  on  the  acquisition  workforce,  “The 
Defense  Acquisition  Workforce  Improvement  Act  (DAWIA)  of  1990  addressed  the 
need  to  improve  the  quality  of  the  acquisition  workforce,  establishing  formal  career 
paths  and  standards  for  education  and  training”  (Eide  &  Allen,  2012,  p.  102). 

President  Clinton  signed  two  laws  focused  on  acquisition  reform  during  his 
presidency:  the  Federal  Acquisition  Streamlining  Act  (FASA)  of  1994  and  the 
Clinger-Cohen  Act  of  1 996.  To  broaden  the  definition  of  what  a  commercial  item 
was,  “FASA  streamlined  procurement  of  commercial  items  by  exempting  them  from 
existing  laws”  (Eide  &  Allen,  2012,  p.  102).  The  Clinger-Cohen  Act  of  1996 
facilitated  the  way  that  the  government  acquires  information  technology  by 
“eliminat[ing]  cost  accounting  standards  that  had  discouraged  commercial 
companies  from  doing  business  with  the  federal  government”  (Eide  &  Allen,  2012,  p. 
102). 

In  2005,  the  Defense  Acquisition  Performance  Assessment  Project  was 
established  to  provide  an  assessment  of  the  DoD’s  acquisition  system.  Major 
findings  of  the  project  included  identification  of  “excessive  oversight  and  complex 
acquisition  processes  as  cost  and  schedule  drivers,  and  called  for  stability  of 
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requirements  as  an  essential  element  for  an  effective  acquisition  system”  (Eide  & 
Allen,  2012,  p.  104). 

The  WSARA  aimed  at  improving  the  success  of  major  defense  acquisition 
programs  (MDAPs).  To  accomplish  this  task,  the  WSARA  provided  focus  on  early 
decision  variables  such  as  “reliable  and  independent  baseline  cost  estimates, 
rigorous  early  developmental  testing  and  systems  engineering  oversight,  and  strong 
gatekeeping  to  prevent  programs  from  proceeding  with  too  much  risk  of  immature 
technology”  (Eide  &  Allen,  2012,  p.  106).  Along  with  program  success,  an  implied 
goal  of  the  WSARA  was  to  reduce  costs  and  risk.  Increased  competition  was 
incorporated  in  the  WSARA  to  help  accomplish  this  goal  (Eide  &  Allen,  2012,  p.  106). 

The  WSARA’s  key  provisions  altered  the  DoD’s  acquisition  bureaucracy  in 
addition  to  process  systems  and  senior  management  oversight  changes  (Berteau, 
Hofbauer,  &  Sanok,  2010,  p.  4).  The  WSARA  also  legislated  important  changes  on 
acquisition  processes.  According  to  Berteau  et  al.,  the  WSARA 

•  increased  competition  throughout  the  acquisition  process, 

•  improved  requirement  formulation  processes, 

•  improved  cost-estimation  processes, 

•  enforced  a  more  stringent  set  of  regulations  on  organizational  conflicts 
of  interest, 

•  revised  Milestone  A  and  Milestone  B  certification  processes, 

•  revised  processes  for  reporting  critical  cost  growth,  and 

•  increased  congressional  oversight  through  heightened  reporting 
requirements. 

The  WSARA  created  four  senior-level  Office  of  the  Secretary  of  Defense 
positions  with  military-service-level  equivalents  for  the  DoD  (Berteau  et  al.,  2010; 
WSARA  of  2009): 

•  Director  of  Cost  Assessment  &  Program  Evaluation  (DCAPE) 

•  Director,  Development  Test  &  Evaluation  (DT&E) 

•  Director,  Systems  Engineering  (SE) 

•  Director  for  Performance  Assessments  and  Root  Cause  Analyses 
(PARCA) 

The  primary  responsibilities  of  these  offices  are  illustrated  in  Figure  5. 
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Table  2:  Key  Responsibilities  of  Offices  Established  as  a  Result  of  the  Reform  Act 

Office 

Primary  responsibilities 

Systems  Engineering 

serves  as  principal  advisor  to  the  Secretary  of  Defense  and  the  Under  Secretary  of  Defense 

for  Acquisition.  Technology  and  Logistics  on  systems  engineering  activities  In  the  department 

develops  systems  engineering  and  development  planning  guidance  for  DOD 

reviews  and  approves  major  defense  acquisition  program  systems  engineering  plans 

monitors  major  defense  acquisition  program  systems  engineering  activities 

reports  to  Congress  annually  on  systems  engineering  organization,  capabilities,  and  activities 

Developmental  Test  and 
Evaluation 

serves  as  principal  advisor  to  the  Secretary  of  Defense  and  the  Under  Secretary  of  Defense 
for  Acquisition.  Technology  and  Logistics  on  developmental  test  and  evaluation  activities 
develops  developmental  test  and  evaluation  guidance  for  DOD 

reviews  and  approves  major  defense  acquisition  program  developmental  test  and  evaluation 
plans 

monitors  and  reviews  acquisition  program  developmental  test  and  evaluation  activities  of 
major  defense  acquisition  programs 

reports  to  Congress  annually  on  developmental  test  and  evaluation  organization,  capabilities, 
and  activities 

Cost  Assessment  and  Program 
Evaluation 

serves  as  principal  advisor  to  the  Secretary  of  Defense  and  other  senior  officials  on  matters 
related  to  cost  analysis  and  the  planning  and  programming  phases  of  the  planning, 
programming,  budgetirrg.  and  execution  system 

develops  independent  cost  estimates  for  major  defense  acquisition  programs  prior  to  major 
milestone  decisions  and  updates  independent  cost  estimates  when  programs  have  exceeded 
critical  cost  thresholds,  known  as  Nunn  McCurdy  breaches 

reviews  existing  systems  artd  methods  for  tracking  and  assessing  operation  and  support 
costs  on  major  defense  acquisition  programs 

develops  analysis  of  alternative  study  guidance  for  major  defense  acquisition  programs 
approves  the  analysis  of  alternatives  study  plan  for  each  major  defense  acquisition  program 

Performance  Assessments  and 
Root  Cause  Analyses 

assesses  major  acquisition  program  performance  through  independent  analyses  and  through 
the  Defense  Acquisition  Executive  Summary  process 

identifies  the  root  causes  of  cost  growth  and  other  problems  on  programs  that  experience  a 
critical  Nunn  McCurdy  cost  breach 

Sourea:  GAO  Analvws  Of  Weapon  SYSleoB  Acquisioon  RetormAci  of  2009. 

Figure  5.  WSARA  Key  Office  Responsibiiities 

(Sullivan,  2012,  p.  5) 

2.  WSARA  Implementation  and  Correlation  to  Program  and  Product 
Success 

Finding  the  relationship  between  a  legislation’s  intended  effects  and  its  actual 
results  on  the  group  to  be  legislated  is  challenging.  Legislation  rarely  has  a  direct 
visual  result  on  the  DoD’s  bottom  line  but  rather  creates  a  waterfall  of  changes  and 
secondary  effects.  A  correlation  between  two  variables  can  exist,  but  causation  is  a 
different  matter.  In  our  research  of  the  WSARA  law  and  its  provisions,  we  make  an 
assumption  that  the  WSARA  is  the  initiator  of  bottom-line  change,  if  not  the  catalyst 
for  changes  that  occur. 

The  bottom  line  in  the  corporate  world  is  profit.  For  a  governmental  entity  like 
the  DoD,  the  acquisition  bottom  line  is  measured  in  cost,  schedule,  and  performance 
There  is  a  correlation  between  other  acquisition  initiatives  and  cost,  schedule,  and 
performance  metrics.  Biery  (1992),  along  with  some  other  researchers,  argued  that 
the  DoD  was  doing  well  during  the  1990s,  positing  that  “if  meeting  cost  and  schedule 
objectives  measures  an  institution’s  managerial  efficiency,  then  the  DOD  is  more 
efficient  than  the  typical  commercial  business  or  government  organization”  (p.  644). 
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Biery  (1992)  also  illustrated  that  the  end  of  each  decade  has  a  decreasing  trend  of 
cost  growth  and  compound  annual  growth  rate. 

Countering  Biery’s  (1992)  argument,  Christensen  et  al.  (1999)  conducted  a 
cost  overrun  analysis  on  269  contracts  that  were  four  years  before  and  four  years 
after  the  implementation  of  the  Packard  Commission  recommendations.  The 
assumption  was  that  there  would  be  a  clear  drop  in  cost  overruns.  However, 
Christensen  et  al.’s  (1999)  results,  as  seen  in  Figure  6,  “show  that  the  Packard 
Commission’s  recommendations  did  not  improve  the  cost  performance  of  defense 
acquisition  contracts”  (p.  256). 


Cost  Performance 


Contracts 


Figure  6.  Graph  From  Cost  Overrun  Analysis  on  Packard  Commission 

Reforms 

(Christensen  et  al.,  1999) 

The  WSARA’s  implementation  has  had  success,  as  illustrated  in  the  third 
GAO  report  on  the  WSARA,  authored  by  Sullivan  (2012).  According  to  the  report, 
the  “GAO’s  analysis  of  1 1  weapon  acquisition  programs  showed  the  Reform  Act  has 
reinforced  early  attention  to  requirements,  cost  and  schedule  estimates,  testing,  and 
reliability”  (Sullivan,  2012,  p.  1).  This  is  reinforced  by  GAO  Report  No.  GAO-1 4- 
145T,  authored  by  Francis  (2013);  Francis  indicated  that  “to  the  extent  reforms  like 
the  Weapon  Systems  Acquisition  Reform  Act  and  DOD’s  Better  Buying  Power 
initiatives  are  being  implemented,  they  are  having  a  positive  effect  on  individual 
programs”  (p.  5). 
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3. 


Desired  Effects  From  the  WSARA 


The  Department  of  Defense’s  Rapid  Acquisition  Process:  Is  It  a  Model  for 
Improving  Acquisition?,  the  first  hearing  before  the  Panel  on  Defense  Acquisition 
Reform  of  the  House  Armed  Services  Committee  on  June  3,  2009,  highlighted 
ongoing  challenges  associated  with  the  DAS.  During  the  hearing,  Gordon  England, 
a  former  deputy  secretary  of  defense  and  president  of  the  company  E6  Partners, 
spoke  on  the  complexity  of  the  DAS: 

Counterintuitively,  that  means  you  want  to  give  managers  more 
flexibility.  The  more  complex  the  system,  the  more  flexibility  you  need, 
managers  need.  The  trend  is  always  the  other  way.  That  is  it  gets  more 
complex,  we  add  layers  of  bureaucracy  and  regulation  and  control  and 
that  makes  it  almost  impossible  to  run  very  complex  programs.  So  the 
system  today  is  way  overburdened.  It  is  over-burdened  by  the 
Department,  it  is  overburdened  by  the  Congress.  As  it  becomes  a 
more  complex  system,  we  need  to  simplify  it,  otherwise  managers 
won't  be  able  to  operate.  {The  Department  of  Defense’s  Rapid 
Acquisition  Process,  2009,  p.  6) 

Representative  Jim  Cooper  from  Tennessee  mentioned  additional  complexity 
added  to  the  acquisition  process  from  the  WSARA  but  unfortunately  suggested  that 
no  one  read  the  law,  stating, 

I  am  worried  about  the  Tower  of  Babel  effect  when  we  create  a  system 
that  is  so  complex  that  nobody  can  understand  it.  We  were  just  joking 
prior  to  the  hearing  that  how  many  people  actually  read  the  weapons 
acquisition  bill  that  we  just  passed.  Nobody.  {The  Department  of 
Defense’s  Rapid  Acquisition  Process,  2009,  p.  17). 

Robert  Andrews,  the  chairman  of  the  panel  at  the  time,  addressed  the  lack  of 
coordination  between  requirements,  budgeting,  and  procurement  {The  Department 
of  Defense’s  Rapid  Acquisition  Process,  2009,  p.  10).  Luckily,  the  WSARA 
addressed  some  of  Andrews’  concerns. 

Programs  are  just  beginning  to  implement  the  provisions  of  the  WSARA 
(Sullivan,  2012).  The  report  points  to  little  to  no  improvement  in  acquisition  areas. 
However,  according  to  a  2013  report  by  the  Government  Business  Council,  there 
has  been  improvement  in  personal  accountability  (as  shown  in  Figure  7). 
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Figure  7.  Acquisition  Areas  of  Improvement,  Affirmative  Answers 

(Government  Business  Council,  2013,  p.  21) 


4.  Conciusion 

This  chapter  introduced  past  and  current  literature  on  the  establishment  and 
implementation  of  the  WSARA;  correlation  between  WSARA  implementation  and 
program  and  product  successes;  intended  effects  of  the  WSARA;  cost,  schedule, 
and  performance  and  contract  metrics;  and  effects-based  management.  The  next 
chapter  describes  research  methodology,  our  data  collection  process,  a  description 
of  the  Program  Executive  Office  for  Combat  Support  and  Combat  Service  Support 
(PEO  CS&CSS),  a  list  of  our  survey  questions,  and  the  analysis  we  conducted. 

The  literature  covering  this  topic  has  several  agreements.  First,  consistent 
acquisition  reform  is  needed  as  requirements  are  continuously  generated  from 
threat-based  analysis  within  budgetary  constraints;  refinement  and  reform  of  the 
acquisition  system  is  needed.  Second,  acquisition  reform  is  ongoing.  As  mentioned, 
threats  and  the  operating  environment  are  always  changing.  Therefore,  timely  and 
accurate  reform  is  needed  in  legislation  and  policies  to  develop  the  right  solution  at 
the  right  time.  Third,  flexibility  is  needed  in  a  complex  system.  The  ability  to  provide 
resources  to  the  right  people  to  execute  solutions  in  a  complex  system  is  paramount. 
Fourth,  more  accurate  metrics  are  needed  on  cost,  performance,  and  schedule  to 
provide  senior  leadership  with  the  right  information  to  make  decisions. 

The  literature  covering  this  topic  has  a  couple  of  disagreements.  First, 
several  of  the  authors  at  different  points  in  time  (both  current  and  past)  have 
disagreed  as  to  whether  acquisition  reform,  the  WSARA  included,  is  doing  a  good 
job.  Flindsight  is  20/20 — it  is  easier  to  measure  reforms  1 0  years  after  they  have 
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been  implemented  and  the  entire  operating  environment  has  changed.  Second, 
there  is  disagreement  on  how  success  is  measured.  Managers  at  the  program  and 
product  levels  have  a  different  interpretation  of  which  metrics  should  be  used  versus 
those  at  Congress  or  in  the  private  sector. 

Our  project  covers  a  specific  case  study.  We  focus  on  the  PEO  CS&CSS  and 
its  products.  By  focusing  on  a  single  office  and  seeing  how  the  WSARA  is  being 
implemented  there,  we  can  provide  a  more  accurate  sample  rather  than  providing  a 
blanket  population  or  randomly  selected  programs.  We  can  also  identify  successes 
and  challenges  more  accurately.  The  lessons  learned  at  a  specific  program  office 
could  then  be  utilized  at  other  offices  as  the  personnel  see  fit.  Furthermore,  we  use 
an  analysis  on  effects-based  management  and  how  it  could  be  implemented  into 
different  levels  of  management  to  yield  measurable  results. 
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III.  METHODOLOGY 


A.  INTRODUCTION 

In  this  chapter,  we  address  the  organizational  structure  of  the  PEO  CS&CSS 
and  then  explain  our  interview  and  quantitative  data  methodologies.  We  generated 
interview  questions  and  research  efforts  utilizing  the  primary  and  secondary 
research  questions  for  this  project.  The  project  questions  are  as  follows: 

•  Overarching  project  question:  How  are  WSARA  Title  II  policies  being 
applied  in  major  defense  acquisition  programs  (MDAPs)? 

•  Why  was  the  WSARA  needed,  or  what  were  the  precursors  that  led  to 
the  WSARA? 

•  What  was  the  intended  effect  of  the  WSARA? 

•  How  effective  has  the  implementation  of  the  WSARA  been  in  the  PEO 
CS&CSS? 

•  Are  there  gaps  between  the  intent  and  application  of  Title  II  of  the 
WSARA  within  a  major  PEO?  If  so,  why? 

Our  intent  with  these  questions  was  to  determine  whether  acquisition  reforms, 
such  as  those  included  in  Title  II  of  the  WSARA  of  2009,  have  their  desired  effect  in 
major  acquisition  programs. 

B.  PEO  CS&CSS  LOCATION  AND  ORGANIZATIONAL  STRUCTURE 

The  PEO  CS&CSS  is  located  within  the  Tank  and  Automotive  Command 
(TACOM)  headquarters  perimeter  in  Warren,  Ml,  and  is  one  of  four  PEOs  within  the 
TACOM  structure.  The  PEO  CS&CSS  follows  the  typical  major  acquisition 
organization:  The  PEO  and  his  or  her  deputies  orchestrate  the  operations  of  the 
PMs  (civilian  or  military  members  in  Grade  06  [colonel]  or  the  civilian  equivalent)  and 
product  managers  (PdMs;  civilian  or  military  members  in  Grade  05  [lieutenant 
colonel]  or  the  civilian  equivalent)  and  their  various  assistants  and  staff.  For  the 
PEO  CS&CSS,  the  PM  level  is  broken  down  into  the  following  categories: 

•  Force  protection:  This  category  includes  bridging,  combat  engineering, 
force  sustainment,  petroleum  and  water,  sets,  kits,  outfits  and  tools, 
test  measurement  and  diagnostic  equipment,  and  contingency  basing 
infrastructure. 
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•  The  Joint  Program  Office  for  the  Joint  Light  Tactical  Vehicle  (JLTV): 
This  category  includes  the  JLTV,  the  joint  Army-Marine  acquisition 
effort  currently  in  the  EMD  phase  of  the  acquisition  life  cycle. 

•  Mine-resistant,  ambush-protected  vehicles  (MRAP):  This  category 
includes  MRAP-ATV  systems  and  joint  logistics. 

•  Mobile  electric  power:  This  category  includes  small,  medium,  large, 
and  battery-powered  electrical  power  sources. 

•  Transportation  systems:  This  category  is  composed  of  personnel  and 
logistics  transportation  systems,  including  medium  tactical  vehicles, 
armored  security  vehicles,  heavy  tactical  vehicles,  allied  tactical 
vehicles,  and  Army  watercraft  systems.  (PEO  CS&CSS,  n.d.) 

C.  INTERVIEWEE  SELECTION 

In  late  October  2013,  a  list  of  Army-sponsored  research  projects  was 
distributed  through  the  Naval  Postgraduate  School’s  Graduate  School  of  Business 
and  Public  Policy  channels.  Among  the  projects  on  the  list  was  a  research  project 
sponsored  by  the  PEO  CS&CSS  stating  a  request  to  research  acquisition  reform 
and  determine  whether  reforms  are  implemented  and  the  intended  effects  are  being 
achieved.  In  this  project,  we  focused  the  question  concerning  acquisition  reform  to 
include  the  WSARA  of  2009,  specifically  Title  II.  Title  II  of  the  WSARA  was 
determined  to  be  the  portion  of  the  legislation  that  was  most  readily  translatable  into 
interview  and  research  questions  supporting  the  greater  request  issued  by  the  PEO 
CS&CSS. 

We  began  correspondence  with  the  PEO  CS&CSS-designated  point  of 
contact  to  determine  interviewee  availability.  We  requested  individuals  operating  at 
the  PM  and  PdM  levels  as  the  primary  interviewees,  with  the  understanding  that 
these  personnel  are  ultimately  responsible  for  managing  the  cost,  schedule, 
performance,  and  risk  of  PEO  CS&CSS  programs.  Cost,  schedule,  performance, 
and  risk  are  the  focal  points  of  the  Title  II  WSARA  reforms. 

D.  INTERVIEW  METHOD 

We  conducted  interviews  on-site  with  the  PEO  CS&CSS  in  Warren,  Ml.  We 
utilized  this  methodology  in  order  to  achieve  accurate  and  spontaneous  responses 
from  interviewees.  Rather  than  utilizing  an  e-mailed  survey  or  questionnaire,  which 
potentially  would  allow  interviewees  to  manipulate  or  research  a  best  answer  to  a 
question,  face-to-face  interviews  afforded  honest  and  candid  responses  in  situ. 
Additionally,  conducting  interviews  on-site  allowed  us  to  develop  perceptions 
concerning  the  PEO  CS&CSS  and  its  priorities  that  could  not  be  gleaned  otherwise. 
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E.  INTERVIEW  QUESTION  DEVELOPMENT 

We  developed  the  interview  questions  based  on  Title  II  of  the  WSARA,  as 
well  as  expectancy  management  concepts.  First,  we  converted  each  subsection  of 
the  Title  II  WSARA  legislation  into  the  form  of  a  question.  The  answers  to  the 
question  would  serve  as  a  means  for  qualifying  the  effects  of  the  WSARA  at  the 
PEO  level.  We  then  categorized  the  interview  questions  based  on  the  legislation  in 
order  to  nest  them  for  analysis  of  overarching  project  questions.  Second,  we 
developed  questions  that  focused  on  the  interviewee’s  interpretation  of  the 
effectiveness  of  the  WSARA,  the  incumbent  challenges  as  well  as  successes. 

F.  DEFENSE  ACQUISITION  MANAGEMENT  INFORMATION  RETRIEVAL 
SYSTEM  DATA 

The  Defense  Acquisition  Management  Information  Retrieval  System  (DAMIR) 
compiles  multiple  sources  from  MDAP  information  systems,  as  well  as  major 
automated  information  systems.  According  to  its  website,  DAMIR  is  “the 
authoritative  source  for  Selected  Acquisition  Reports  (SAR),  SAR  Baseline, 
Acquisition  Program  Baselines  (APB),  and  Assessments”  (DAMIR,  2014). 

We  gathered  data  from  DAMIR  with  the  intent  of  conducting  a  high-level 
quantitative  analysis  on  the  data.  We  were  looking  for  a  correlation  between 
program  bottom-line  numbers  and  when  the  WSARA  was  implemented.  We  chose 
SARs  for  all  Army  programs  present  in  the  DAMIR  system  from  2005  to  2012. 
According  to  the  press  release  (DoD,  2013a), 

SARs  summarize  the  latest  estimates  of  cost,  schedule  and 
performance  status.  These  reports  are  prepared  annually  in 
conjunction  with  the  submission  of  the  President’s  Budget.  ...The  total 
program  cost  estimates  provided  in  the  SARs  include  research  and 
development,  procurement,  military  construction,  and  acquisition- 
related  operation  and  maintenance.  Total  program  costs  reflect  actual 
costs  to  date  as  well  as  future  anticipated  costs.  All  estimates  are 
shown  in  fully  inflated  then-year  dollars. 

The  SAR  categories  that  we  collected  categorized  the  Army  programs  by  the 
following: 

•  current  estimate, 

•  contractor, 

•  unit  cost  by  percentage  change, 

•  sunk  funding, 

•  cost  variance,  and 

•  schedule. 
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The  sample  data  had  1 7  programs.  There  was  no  Army  SAR  data  available 
for  calendar  year  (CY)  2008.  We  then  placed  the  data  into  a  pivot  table  and  gave 
the  data  sparklines  to  identify  and  illustrate  trends.  The  dependent  variable  was  the 
report  categories’  cost  or  given  numerical  value.  The  independent  variable  was  the 
SAR  report  calendar  date. 

G.  CONCLUSION 

The  interviews  and  cost  data  were  used  to  gain  a  qualitative  and  quantitative 
perspective  on  the  topic.  The  interviews  provided  a  focused  view  of  the  legislation’s 
impact  on  a  program  office.  The  cost  data  provided  a  global  view  on  the  legislation’s 
effect  on  the  Army’s  MDAP  portfolio. 
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IV.  DATA  AND  ANALYSIS 


A.  INTRODUCTION 

This  chapter  addresses  the  organizational  structure  of  PEO  CS&CSS 
followed  by  an  analysis  of  interview  questions  driven  by  the  primary  and  subordinate 
research  questions  for  this  project.  These  questions  are  as  follows: 

1 .  Overarching  project  question:  How  are  WSARA  Title  II  policies  being 
applied  in  MDAPs? 

2.  Why  was  the  WSARA  needed,  or  what  were  the  precursors  that  led  to 
the  WSARA? 

3.  What  was  the  intended  effect  of  the  WSARA? 

4.  How  effective  has  the  implementation  of  the  WSARA  been  in  PEO 
CS&CSS? 

5.  Are  there  gaps  between  the  intent  and  application  of  Title  II  WSARA 
within  a  major  PEO?  If  so,  why? 

The  intent  is  to  determine  whether  acquisition  reforms,  such  as  those 
included  in  Title  II  of  the  WSARA  of  2009,  have  their  intended  effect  in  major 
acquisition  programs.  On-site  interviews  were  conducted  to  determine  the  extent 
that  the  WSARA  is  understood  and  implemented  in  PEO  CS&CSS.  Additional  data 
were  collected  through  analysis  of  program  data  obtained  from  the  DAMIR  website 
and  other  related  program  documents  for  amplifying  information.  The  interview 
questions  are  grouped  under  the  thesis  questions,  followed  by  our  analysis  of 
responses.  We  omit  the  first  question  in  this  chapter  but  return  to  it  in  the 
conclusions  and  recommendations  chapter. 

B.  PEO  CS&CSS  ORGANIZATIONAL  STRUCTURE 

Major  acquisition  organizations  follow  a  structure  where  the  program 
executive  officer  (PEO)  and  his  or  her  deputies  orchestrate  the  operations  of  the 
program  manager  (PM,  civilian  or  military  member  in  the  grade  of  06  [colonel]  or 
civilian  equivalent)  and  product  manager  (PdM,  civilian  or  military  member  in  grade 
05  [lieutenant  colonel]  or  civilian  equivalent),  with  their  various  assistants  and  staff 
following  this  structure,  as  depicted  in  Figure  8. 
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Figure  8.  Example  PEO  Staff  Structure 

For  PEO  CS&CSS,  the  PM  level  is  broken  down  into  the  following  categories: 

•  Force  Protection:  This  includes  bridging,  combat  engineering,  force 
sustainment,  petroleum  and  water,  sets,  kits,  outfits  and  tools  (SKOT), 
test  measurement  and  diagnostic  equipment,  and  contingency  basing 
infrastructure. 

•  Joint  Program  Office  for  the  Joint  Light  Tactical  Vehicle  (JLTV): 

The  JLTV  is  the  joint  Army-Marine  acquisition  effort  currently  in  the 
Engineering  and  Manufacturing  Development  (EMD)  phase  of  the 
acquisition  life  cycle. 

•  Mine-Resistant  Ambush  Protected  Vehicles  (MRAP):  This  includes 
MRAP-ATV  (MATV)  systems  and  joint  logistics. 

•  Mobile  Electric  Power:  This  category  includes  small,  medium,  large, 
and  battery-powered  electrical  power  sources. 

•  Transportation  Systems:  This  category  is  comprised  of  personnel 
and  logistics  transportation  systems  including  medium  tactical  vehicles, 
armored  security  vehicles,  heavy  tactical  vehicles,  allied  tactical 
vehicles  and  Army  watercraft  systems. 

PEO  CS&CSS  manages  over  350  systems  in  total.  Our  initial  interviews  were 
conducted  with  various  representatives  within  this  organizational  structure. 

1.  Task  Allocation 

WSARA  legislation  called  for  additional  high-level  oversight  at  the  Office  of 
the  Secretary  of  Defense  (OSD).  The  interviews  indicated  that  the  PEO  staff 
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structure  was  in  place  to  match  what  the  WSARA  required  at  OSD  level,  as  shown  in 
Figure  5  of  our  literature  review  section.  Project  and  product  offices  had  teams  or 
individuals  that  would  handle  data  calls  required  by  PEO  staff  or  OSD  level. 
Additionally,  while  the  organization  was  structured  in  accordance  with  regulation, 
PdMs  were  capable  of  adapting  their  structure  based  on  their  program’s  current 
phase  within  the  Integrated  Defense  Acquisition,  Technology,  and  Logistics  Life 
Cycle  Management  System. 

Work  tasks  are  allocated  in  a  hierarchical  fashion  from  PEO  to  project  office 
to  product  office.  At  the  product  office  level,  tasks  are  allocated  by  PdM-identified 
priorities. 

C.  WSARA  QUANTITATIVE  IMPACTS 

The  intended  effect  of  the  WSARA  was  “to  improve  the  organization  and 
procedures  of  the  Department  of  Defense  for  the  acquisition  of  major  weapon 
systems,  and  for  other  purposes”  (WSARA  of  2009,  2009),  with  the  secondary  intent 
to  reduce  risk  and  costs.  To  accomplish  this,  the  legislation  focused  on  changing 
senior  management  oversight  and  improving  early  decision  variables  in  the 
acquisition  system. 

As  established  in  the  literature  review,  the  need  for  the  law  was  clear.  Cost 
overruns  were  ever  increasing  in  frequency  and  value.  There  was  a  recognized  need 
“to  improve  the  way  weapon  systems  are  acquired”  (Sullivan,  2012,  p.  4).  The  intent 
of  the  law  was  to  limit  those  overruns  and  enhance  oversight  and  accountability  by 
adding  additional  layers  of  oversight  and  streamlining  the  monitoring  process  to 
“ensure  that  costs  are  controlled”  (Sullivan,  2012,  p.  8). 

1.  Defense  Acquisition  Management  Information  Retrievai 

DAMIR  compiles  multiple  sources  from  MDAP  and  major  automated 
information  system  (MAIS)  information  systems.  According  to  its  website,  DAMIR  is 
“the  authoritative  source  for  Selected  Acquisition  Reports  (SAR),  SAR  Baseline, 
Acquisition  Program  Baselines  (APB),  and  Assessments”  (DAMIR,  2013).  We 
pulled  relevant  data  from  DAMIR.  The  data  retrieved  were  from  the  Army’s  2005- 
2012  SARs  for  noted  MDAPs.  These  sources  of  data  are  relevant  as  stated  by  the 
DoD  (2013a): 

SARs  summarize  the  latest  estimates  of  cost,  schedule  and 
performance  status.  These  reports  are  prepared  annually  in 
conjunction  with  the  submission  of  the  President’s  Budget. 

Subsequent  quarterly  exception  reports  are  required  only  for  those 
programs  experiencing  unit  cost  increases  of  at  least  15  percent  or 
schedule  delays  of  at  least  six  months. 
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The  total  program  cost  estimates  provided  in  the  SARs  include 
research  and  development,  procurement,  military  construction,  and 
acquisition-related  operation  and  maintenance.  Total  program  costs 
reflect  actual  costs  to  date  as  well  as  future  anticipated  costs.  All 
estimates  are  shown  in  fully  inflated  then-year  dollars. 

The  reports  were  chosen  because  they  reflect  the  most  relevant  data  on  the 
Army’s  MDAPs  from  2005  to  2012.  Those  years  were  selected  to  analyze  changes 
occurring  in  the  period  four  years  before  the  WSARA  and  three  years  after.  The 
intent  of  analyzing  the  years  before  and  after  the  WSARA  was  implemented  to  see 
any  changes  within  the  sample  on  cost  (which  was  the  WSARA’s  secondary 
objective  and  also  the  best  quantitative  measurement  for  success).  There  were  no 
Army  SARs  listed  in  DAMIR  for  CY2008. 

2.  Trends  in  Cost  Data 

We  specifically  analyzed  three  quantitative  cost  measures  to  see  whether 
there  were  any  trends:  current  estimate  total  cost;  program  acquisition  unit  cost 
(PAUC)  percent  change  compared  to  the  baseline  estimate;  and  average 
procurement  unit  cost  (APUC)  percent  change  compared  to  the  baseline  estimate. 
We  were  looking  to  see  whether  costs  were  increasing,  decreasing,  or  varied  before 
and  after  implementation  of  the  WSARA.  In  addition,  we  wanted  to  see  whether 
there  was  variation  of  cost  changes  before  and  after  implementation  of  the  WSARA. 
Figures  9  through  1 1  from  our  quantitative  analysis  show  the  trends  in  available  data 
for  the  programs  based  off  the  current  estimate,  PAUC,  and  APUC  for  each 
program: 
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SAR  Current  Estimate  Tot< 

)l  Cost  in 

Millions 

TRENDLINE 

PROGRAM 

2005 

2006 

2007 

2009 

2010 

2011 

2012 

1  ^ 

1  X 

1 

AH-64E  New 

Build 

2352.7 

2155.8 

2484.5 

AH-64E 

Remanufacture 

8990.4 

8996.4 

11686 

11893 

11968 

13760 

AMFJTRS 

9067.1 

9005.7 

4229.5 

3759.9 

CH-47F 

12798 

13449 

13350 

13636 

14439 

14257 

14387 

Excalibur 

2292.7 

2302.8 

2465.1 

2469.6 

1710.6 

1676.7 

1697.8 

1   ^ 

GMLRS/GMLRS 

AW 

16035 

6772.5 

6008.3 

6058.9 

6024.4 

6411.5 

6693.9 

lAMD 

5791.6 

6320.4 

6694.4 

6375.2 

JLENS 

7340.9 

7367.3 

7500.3 

8063.8 

8537.9 

2620.5 

2645.8 

JTN 

1786.6 

2104.1 

1961.8 

1939 

1988.4 

1992.6 

2084.3 

JTRSHMS 

11599 

11789 

3366.9 

5240.4 

5811.4 

9304.7 

10191 

MQ-lCGray 

Eagle 

5220.8 

5262.5 

4745.3 

4888.9 

PAC-3 

8527.7 

8521.7 

8524.9 

9326.1 

10002 

10797 

11007 

•  TV— 

Patriot/MEADS 

CAP 

30531 

30178 

29895 

30827 

12563 

12952 

12832 

i  / 

PIM 

7899.3 

7904.2 

UH-60M  Black 

Hawk 

23770 

23830 

24043 

23682 

27341 

28861 

25694 

WIN-Tlnc2 

3870.8 

4997.7 

6352.5 

6461.3 

5137.4 

WIN-Tlnc3 

15964 

16056 

14456 

17890 

Grand  Total 

114680 

115304 

109983 

153970 

145660 

147482 

149433 

Figure  9.  SAR  Current  Estimate,  Totai  Cost  in  Miiiions  ($) 
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SAR  PAU( 

1  %  Change  > 

\galnst  Orig 

nal  Estimate 

Baseline 

TRENDLINE 

PROGRAM 

2005 

2006 

2007 

2009 

2010 

2011 

2012 

1  \ 

1  X  ^ 

1 

AH-64E  New  Build 

-0.74% 

-13.78% 

-8.11% 

AH-64E  Remanufacture 

2.33% 

2.92% 

28.41% 

-0.15% 

-2.09% 

10.53% 

1  ^ 

1* — \ 

1  \ 

1  X 

AMFJTRS 

2.45% 

1.05% 

6.99% 

-27.62% 

1 

CH-47F 

3.18% 

8.59% 

8.28% 

10.73% 

15.17% 

13.10% 

13.48% 

-I*-- 

i  *  '  , 

Excalibur 

-4.62% 

-4.62% 

1.39% 

4.17% 

1.81% 

-0.90% 

-4.07% 

i 

X—}-  - - 

GMLRS/GMLRS  AW 

17.14% 

68.57% 

2.86% 

6.67% 

5.71% 

9.52% 

12.38% 

lAMD 

0.00% 

11.09% 

-26.33% 

-26.58% 

JLENS 

2.20% 

1.36% 

4.08% 

12.35% 

17.88% 

215.72% 

217.46% 

1 

1 

1 

JTN 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

■  '"i. . --- 

JTRS  HMS 

3.85% 

3.85% 

7.69% 

-26.92% 

-19.23% 

0.00% 

3.33% 

MO-IC  Gray  Eagle 

0.00% 

-4.38% 

-14.53% 

-12.74% 

1 

PAC-3 

25.03% 

23.47% 

23.64% 

16.99% 

13.31% 

12.67% 

9.37% 

Patriot/MEADS  CAP 

-5.37% 

-7.76% 

-7.47% 

3.97% 

8.94% 

10.44% 

9.11% 

1  \ 

PIM 

0.00% 

-1.47% 

---i - 

UH-60M  Black  Hawk 

9.61% 

9.08% 

12.69% 

16.30% 

18.07% 

22.20% 

8.69% 

WIN-T  Inc  2 

0.00% 

10.68% 

8.32% 

8.63% 

13.40% 

r'^v,- 

WIN*Tlnc3 

-8.77% 

-8.00% 

-14.10% 

-11.26% 

Figure  10.  SAR  PAUC  %  Change  Against  Originai  Baseiine 
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SAR  APUC  %  Change  Against  Original  Estimate  Baseline 


TRENDLINE 

PROGRAM 

2005 

2006 

2007 

2009 

2010 

2011 

2012 

•  *  • 

1  \ 

1  ' 

1  V' 

AH-64E  New  Build 

0.00% 

0.00% 

0.00% 

0.00% 

-0.74% 

-13.78% 

-8.11% 

AH-64E  Remanufacture 

0.00% 

4.07% 

4.65% 

36.49% 

-0.01% 

-2.46% 

13.74% 

•  ♦  ♦ 1  •  •k 

1  .. 

1  \ 

!  N, 

AMFJTR5 

0.00% 

0.00% 

0.00% 

1.36% 

0.00% 

-23.18% 

-53.64% 

CH-47F 

3.20% 

8.70% 

8.38% 

10.91% 

15.45% 

13.34% 

13.72% 

•  I 

Excalibur 

0.00% 

0.00% 

15.38% 

20.51% 

5.26% 

0.00% 

-5.26% 

✓A  1 

"  '  \  4-  . - 

GMLRS/GMLRS  AW 

19.70% 

59.09% 

3.30% 

4.40% 

4.40% 

7.69% 

10.99% 

.  .  .  -U—  -< 

j 

lAMD 

0.00% 

0.00% 

0.00% 

0.00% 

2.20% 

-30.80% 

-37.76% 

JLENS 

2.15% 

1.22% 

5.17% 

11.53% 

13.31% 

0.00% 

0.00% 

1 

1 

• - • — • — r - - — ♦ 

JTN 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

'  ‘  .  i 

JTRS  HMS 

0.00% 

0.00% 

-20.00% 

-36.00% 

-36.00% 

-7.14% 

-3.57% 

•  •  ♦  . 

i  \ . 

MQ-lCGray  Eagle 

0.00% 

0.00% 

0.00% 

0.00% 

-1.62% 

-8.22% 

-6.18% 

PAC-3 

25.35% 

24.20% 

24.52% 

23.74% 

24.11% 

27.49% 

24.74% 

Patriot/MEADSCAP 

-5.29% 

-6.66% 

-6.46% 

3.84% 

6.23% 

7.03% 

5.68% 

PIM 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

-1.67% 

UH-60M  Black  Hawk 

9.70% 

9.21% 

12.88% 

16.48% 

19.02% 

23.31% 

9.22% 

• — • — ^  1 

WIN-Tlnc2 

0.00% 

0.00% 

0.00% 

11.13% 

9.39% 

9.50% 

13.68% 

WIN-Tlnc3 

0.00% 

0.00% 

0.00% 

-7.97% 

-7.45% 

-15.92% 

-10.82% 

Figure  11.  SAR  APUC  %  Change  Against  Original  Baseline 
D.  CENTRAL  TENDENCY 

To  understand  variation  in  cost  measure,  we  analyzed  three  measures  of 
variability  in  central  tendency.  The  intent  was  to  find  the  measurement  that  provides 
the  best  representation  of  the  data.  We  used  the  SARs  to  gather  data  on  the 
standard  deviations,  averages,  and  coefficient  of  variation  (CV)  for  the  specified  cost 
element  current  estimate,  PAUC,  and  APUC  %  changes).  First,  averages  (means) 
were  used  to  gauge  the  total  average  of  each  program.  Second,  standard  deviation 
was  used  to  view  the  average  deviation  of  values  from  the  mean.  Third,  CV  was 
used  to  see  the  dispersion  of  data  relative  to  the  mean  because  it  allows  for  analysis 
of  variation  between  the  programs  that  have  significant  cost  differences. 

The  data  provided  by  DAMIR  had  17  Army  MDAP  programs  listed  from  2005 
to  2012.  Data  taken  as  “before  WSARA”  was  from  2005  to  2009  SARs.  Data  taken 
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as  “after  WSARA”  was  from  2010  to  2012  SARs.  For  averages,  standard  deviation, 
and  coefficient  of  variation,  the  data  were  taken  from  each  program’s  SAR  before 
and  after  the  WSARA,  respectively. 

Table  1  summarizes  the  data  collected  and  calculated;  the  details  are 
discussed  in  the  following  three  paragraphs.  Dollar  values  are  in  millions.  The 
percent  changes  for  PAUC  and  APUC  are  the  change  compared  to  the  original 
baseline  estimate  from  the  SAR. 


Table  1.  SAR  Measures  of  Central  Tendency 


AVERAGES 

Current 

PAUC  % 

APUC% 

Estimate 

Change 

Change 

Programs  that  increased 

11 

5 

5 

Average  increase  before  WSARA 

$10,210.16 

5.16% 

8.69% 

Average  increase  after  WSARA 

$10,370.26 

40.04% 

14.79% 

Average  increase  net 

$160.09 

34.88% 

6.10% 

Programs  that  decreased 

6 

12 

12 

Average  decrease  before  WSARA 

$10,552.49 

4.72% 

3.13% 

Average  decrease  after  WSARA 

$6,014.36 

-2.73% 

-6.02% 

Average  decrease  net 

$(4,538.13) 

-7.45% 

-9.15% 

Aii  Programs  average  before 

WSARA 

$10,347.09 

4.87% 

4.98% 

Aii  Programs  average  after  WSARA 

$8,832.88 

9.85% 

0.10% 

All  Programs  average  net  change 

$(1,514.21) 

4.98% 

-4.06% 

STANDARD  DEVIATIONS 


Current 

Estimate 

PAUC  % 
Change 

APUC% 

Change 

Programs  that  Increased 

9 

8 

10 

stdev  increase  before  WSARA 

$297 

0.0418 

0.0308 

stdev  increase  after  WSARA 

$1,198 

0.2221 

0.0979 

stdev  increase  net 

$901 

0.1803 

0.0671 

Programs  that  decreased 

8 

9 

7 

stdev  decrease  before  WSARA 

$1,572 

0.0953 

0.0963 

stdev  decrease  after  WSARA 

$601 

0.0357 

0.0308 

stdev  decrease  net 

$(971) 

(0.0596) 

(0.0655) 

Aii  Programs  stdev  before  WSARA 

$1,224 

0.0856 

0.0578 

Aii  Programs  stdev  after  WSARA 

$917 

0.1234 

0.0702 

All  Programs  stdev  net  change 

$(307) 

0.0379 

0.0125 

riLAtSWTIANR$CI[,Vf;4^, 
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COEFFICIENT  OF  VARIATION 


Current 

Estimate 

PAUC  % 
Change 

APUC% 

Change 

Programs  that  increased 

10 

7 

10 

CV  increase  before  WSARA 

2.76% 

58.92% 

68.29% 

CV  increase  after  WSARA 

16.40% 

117.74% 

101.69% 

CV  increase  net 

13.64% 

58.82% 

33.40% 

Programs  that  decreased 

7 

7 

5 

CV  decrease  before  WSARA 

22.50% 

226.34% 

124.99% 

CV  decrease  after  WSARA 

8.25% 

95.22% 

39.65% 

CV  decrease  net 

-14.24% 

-131.12% 

-85.34% 

Aii  Programs  CV  before  WSARA 

15.32% 

176.11% 

99.79% 

Aii  Programs  CV  after  WSARA 

13.04% 

106.48% 

81.01% 

All  Programs  CV  net  change 

-2.27% 

-69.63% 

-18.78% 

1.  SAR  Current  Estimate 

The  first  cost  estimate  was  the  SAR  “current  estimate,”  which  is  the  most 
recent  estimate  of  the  program’s  parameters  reflecting  the  President’s  Budget 
proposal,  as  well  as  unforeseen  circumstances  or  unavoidable  circumstances  that 
adjusted  cost  measures  (Hagan,  2012,  p.  B-55). 

Before  WSARA  implementation,  1 1  programs  increased  in  dollar  value  from 
the  current  estimate.  The  average  value  of  the  1 1  programs  before  the  WSARA  was 
$1 0.21  billion.  The  average  cost  estimate  of  the  1 1  programs  after  the  WSARA  was 
$1 0.37  billion,  yielding  a  net  increase  of  $1 60  million.  After  WSARA  implementation, 
six  programs  decreased  in  dollar  value  from  the  current  estimate.  The  average 
value  of  the  six  programs  before  the  WSARA  was  $1 0.55  billion.  The  average  value 
of  the  six  programs  after  the  WSARA  was  $6.01  billion,  yielding  a  decrease  of  $4.54 
billion.  The  average  of  all  1 7  programs  before  the  WSARA  was  $1 0.35  billion.  After 
the  WSARA,  the  average  was  $8.83  billion,  yielding  a  net  decrease  of  $1 .51  billion. 

The  1 1  programs  that  increased  only  increased  a  very  small  amount.  The  six 
programs  decreased  nearly  three  times  over  the  average  decrease  of  the  entire 
Army  portfolio.  However,  this  could  only  be  an  indicator  or  program  cancellations,  or 
appropriations  funding  being  cut.  Therefore,  the  dollar  value  is  not  a  good  indicator 
because  of  funding  types  and  varying  prices  of  programs  within  the  portfolio. 

From  before  and  after  WSARA  implementation,  the  standard  deviation  of  nine 
programs  increased  from  $297.38  million  to  $1 ,1 98.02  million.  Eight  programs’ 
standard  deviations  decreased  from  $1 ,572  million  to  $601  million.  The  combined 
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standard  deviation  difference  between  programs  before  and  after  WSARA 
implementation  indicated  a  $307  million  net  decrease. 

The  standard  deviation  is  a  slightly  better  metric  for  interpreting  current 
estimate  changes.  The  programs  that  increased  did  so  four  times  in  magnitude, 
indicating  that  programs  that  did  increase  in  standard  deviation  are  becoming  more 
spread  out  from  the  average.  The  programs  that  decreased  did  so  by  2.6  times  in 
magnitude,  showing  a  slightly  less  dispersion  around  the  mean.  When  comparing  all 
programs,  they  indicate  less  dispersion  but  only  a  slight  decrease. 

From  before  and  after  WSARA  implementation,  the  CV  of  10  programs 
increased  from  2.76%  to  1 6.40%.  Seven  programs’  CVs  decreased  from  22.5%  to 
8.25%  after  the  WSARA  was  implemented,  a  decrease  of  14.25%.  The  average  of 
all  programs’  CV  went  from  1 5.32%  before  to  1 3.04%  after  the  WSARA,  yielding  a 
decrease  of  2.27%. 

As  stated  before,  CV  shows  the  dispersion  of  data  relative  to  the  mean 
because  it  allows  for  analysis  of  variation  while  eliminating  the  significant  cost 
differences  and  providing  the  answer  in  percent  form.  The  current  estimate  overall 
decrease  is  not  very  significant.  The  program  implication  is  that  current  estimates 
have  become  more  in  control  by  2.27%. 

2.  PAUC  Percent  Change 

PAUC  is  the  program  acquisition  unit  cost.  It  is  computed  by  “dividing  the 
Program  Acquisition  Cost  by  the  Program  Acquisition  Quantity”  (Hagan,  2012,  p.  B- 
1 77).  It  is  basically  how  much  everything  costs  divided  by  the  amount  of  units  that 
are  (going  to  be)  purchased.  We  compared  the  PAUC  percent  change  from  the 
original  baseline  estimate  per  each  program  SAR. 

From  before  and  after  WSARA  implementation,  five  programs’  PAUC 
increased  from  5.1 6%  to  40.04%,  a  net  increase  of  34.88%.  12  programs’  PAUC 
percentage  decreased  from  4.72%  to  -2.73%,  a  net  decrease  of  -7.45%.  The 
combined  average  for  all  17  programs’  PAUC  percentage  change  against  the 
baseline  estimate  increased  from  4.87%  before  the  WSARA  to  9.85%  after,  a  net 
increase  of  4.98%. 

From  before  and  after  WSARA  implementation,  the  standard  deviation  of 
eight  programs  increased  from  0.0418  before  to  0.2221  after,  a  net  increase  of 
0.1 803.  Nine  programs  decreased  from  0.0953  before  to  0.0357,  a  net  decrease  of 
0.0596.  The  combined  standard  deviation  average  increased  from  0.0856  to 
0.1234,  a  net  increase  of  0.0379. 

From  before  and  after  WSARA  implementation,  the  CV  of  seven  programs 
increased  from  58.92%  to  1 1 7.74%,  an  increase  of  58.82%.  Seven  programs 
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decreased  from  226.34%  to  95.22%,  a  decrease  of  1 31 .1 2%.  Two  programs  did  not 
have  data  to  allow  CV  calculation  before  the  WSARA.  The  combined  average  for  14 
programs  with  available  data  to  calculate  CV  yielded  a  1 76.1 1  %  before  to  1 06.48% 
after,  a  net  decrease  of  69.63. 

The  programs  PAUC  that  did  only  increase  did  so  by  34.88%.  Compared  to 
the  standard  deviation  (.1803)  of  increasing  programs,  it  is  a  clear  indicator  of 
programs  becoming  more  expensive.  However,  although  this  is  highlighted  with  the 
CV  of  58.82%,  the  programs  that  decreased  did  so  significantly  by  131 .12%. 

3.  APUC  Percent  Change 

APUC  is  average  procurement  unit  cost.  It  is  “calculated  by  dividing  total 
procurement  cost  by  the  number  of  articles  to  be  procured”  (that  is,  recurring  and 
nonrecurring  costs  associated  with  production  of  the  item  (Hagan,  2012,  p.  B-19).  It 
is  basically  the  costs  associated  with  physically  making  the  product  or  item.  We 
compared  the  APUC  percent  change  from  the  original  baseline  estimate  per  each 
program  SAR. 

From  before  and  after  WSARA  implementation,  five  programs  increased  from 
8.69%  to  1 4.79%,  an  increase  of  6.1  %.  1 2  programs  decreased  from  3.13%  to 
-6.02%,  a  decrease  of  9.1 5%.  The  combined  average  change  in  APUC  decreased 
from  4.98%  before  to  0.1 0%  after,  a  net  decrease  of  4.06%. 

From  before  and  after  WSARA  implementation,  10  programs’  standard 
deviations  increased  from  0.0308  to  0.0979,  an  increase  of  0.0671 .  Seven 
programs  decreased  from  0.0963  to  0.0308,  a  decrease  of  0.0655.  The  combined 
APUC  standard  deviation  increased  from  0.0578  to  0.0702,  a  net  increase  of  0.01 25. 

From  before  and  after  WSARA  implementation,  the  CV  of  10  programs 
increased  from  68.29%  to  101 .69%,  an  increase  of  33.40%.  Five  programs 
decreased  from  1 24.99%  to  39.65%,  a  decrease  of  85.34%.  The  combined  average 
CV  decreased  from  99.79%  to  81 .01  %,  a  net  decrease  of  1 8.78%.  Two  programs 
did  not  have  data  to  allow  CV  calculation  before  the  WSARA. 

Average  procurement  unit  costs  are  decreasing  on  average.  The  standard 
deviation  increase  shows  that  they  are  becoming  more  difficult  to  control;  however, 
the  CV  indicates  that  the  APUC  for  the  Army  portfolio  is  decreasing. 

E.  INTENDED  EFFECTS  OF  THE  WSARA 

This  section  is  based  off  the  interviews  we  conducted.  Each  subsection  of 
the  WSARA’s  Title  II  focused  on  changes  to  current  or  new  acquisition  policy. 

These  questions  were  part  of  the  interview  process  conducted  at  PEO  CS&CSS  to 
determine  whether  and  how  these  legislative  requirements  are  implemented  within 
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this  organization.  Interview  questions  were  primarily  based  on  sections  of  WSARA 
Title  II. 


1.  Consideration  of  Trade-Offs  for  Requirement  Deveiopment 

Section  201  of  the  WSARA  states  that  “the  Secretary  of  Defense  shall  ensure 
that  mechanisms  are  developed  and  implemented  to  require  consideration  of  trade¬ 
offs  among  cost,  schedule,  and  performance  objectives  as  part  of  the  process  for 
developing  requirements  for  Department  of  Defense  acquisition  programs”  with  the 
duty  of  the  MDA  to  ensure  that  “the  program  is  affordable”  (Weapon  System 
Acquisition  Reform  Act  of  2009,  2009). 

Program  requirements  are  generated  at  the  Army  G3/G4  level  and 
disseminated  to  PEOs  and  PMs.  PMs  have  little  say  on  MDAPs  to  control  trade-offs 
among  cost,  schedule,  and  performance  objectives.  However,  the  project  managers 
did  have  influence  on  the  MDA  decision-making  process  on  trade-offs  as  indicated 
by  WSARA  201  (f).  Duties  of  Milestone  Decision  Authority: 

Section  2366b(a)(1)(B)  of  such  title  is  amended  by  inserting 
“appropriate  trade-offs  among  cost,  schedule,  and  performance 
objectives  have  been  made  to  ensure  that”  before  “the  program  is 
affordable.” 

Trade-offs  had  typically  been  made  by  the  time  the  current  project  manager 
was  selected  and  was  actively  managing  a  project.  Objectives  are  typically  already 
set  in  the  program  baseline  CPD.  If  there  are  changes,  project  managers  must 
provide  rationale  for  the  need  to  change  their  baseline  estimates  in  cost,  schedule, 
or  performance. 

Certain  programs  had  less  stringent  trade-off  requirements.  For  example,  the 
MRAP  program  could  change  baseline  estimates  to  meet  dynamic  requirements 
driven  by  the  Global  War  on  Terror  situations  in  Iraq  and  Afghanistan.  As  one  PM 
put  it,  “Cost  was  third,  schedule  was  second,  and  actually  you  balance  it  based  on 
theater  priorities.”  The  Joint  Light  Tactical  Vehicle  (JLTV)  was  a  cost  informed  trade 
driven  platform.  Tradeoff  requirements  were  pushed  to  the  contractor  level  based  off 
of  tiered  capability  development  documents  (CDDs)  and  KPPs. 

The  spirit  of  the  legislation  matches  what  the  PMs  were  doing  with  their 
respective  projects,  depending  on  state  of  the  acquisition  cycle.  WSARA  Section 
201  (a)(2)(A-B)  states  that 

(A)  Department  of  Defense  officials  responsible  for  acquisition,  budget, 
and  cost  estimating  functions  are  provided  an  appropriate  opportunity 
to  develop  estimates  and  raise  cost  and  schedule  matters  before 
performance  objectives  are  established  for  capabilities  for  which  the 
Chairman  of  the  Joint  Requirements  Oversight  Council  is  the  validation 
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authority;  and  (B)  the  process  for  developing  requirements  is 
structured  to  enable  incremental,  evolutionary,  or  spiral  acquisition 
approaches,  including  the  deferral  of  technologies  that  are  not  yet 
mature  and  capabilities  that  are  likely  to  significantly  increase  costs  or 
delay  production  until  later  increments  or  spirals. 

There  was  clear  evidence  that  the  JLTV  program  did  exactly  what  WSARA 
201  (c)  required  of  review  of  joint  military  requirements.  However,  the  older 
programs  that  are  were  already  established  did  not  meet  this  requirement. 

2.  Competition  Poiicy 

The  WSARA  mandates  competition  in  as  many  phases  of  acquisition  as 
possible.  WSARA  (2009)  Section  202  makes  suggestions  for  competition  during 

1 .  competitive  prototyping; 

2.  dual-sourcing; 

3.  unbundling  of  contracts; 

4.  funding  of  next-generation  prototype  systems  or  subsystems; 

5.  use  of  modular,  open  architectures  to  enable  competition  for  upgrades; 

6.  use  of  build-to-print  approaches  to  enable  production  through  multiple 
sources; 

7.  acquisition  of  complete  technical  data  packages; 

8.  periodic  competitions  for  subsystem  upgrades; 

9.  licensing  of  additional  suppliers;  and 

10.  periodic  system  or  program  reviews  to  address  long-term  competitive 
effects  of  program  decisions. 

PEO  CS&CSS  competition  policies  were  assessed  to  determine  the 
strategies  employed  to  ensure  competition  of  products  given  the  stage  of  the  product 
in  the  acquisition  life  cycle.  Reverse  engineering  techniques  are  the  primary 
methods  employed  to  create  their  own  technical  data  package  (TDP),  which  would 
drive  more  competition,  because  the  contractors  put  a  high  cost  on  the  procurement 
of  TDPs.  In  addition,  the  PMs  primarily  used  sole-sourcing  due  to  priorities  and  the 
“finite  amount  of  OEMs  [original  equipment  manufacturers]”  in  the  given  industry  or 
program. 

In  contrast,  the  MRAP  and  JLTV  program  focused  on  values  trade-offs  with 
performance-based  competitions  that  helped  industry  partners  be  successful,  as  well 
as  maintaining  competition  requirements.  The  need  to  develop  the  program 
requirements  and  have  competition  before  Milestone  A  with  defense  acquisition 
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executive  (DAE)  and  component  acquisition  executive  (CAE)  support  would  facilitate 
enforcement. 

3.  Prototyping 

WSARA  Section  203  requires  prototyping  for  MDAPs  before  Milestone  B 
approval.  PEO  CS&CSS  policy  concerning  prototyping,  its  frequency  of  use, 
potential  costs,  and  the  number  of  times  prototyping  is  waived  was  analyzed  to 
determine  the  level  of  compliance  with  this  WSARA  requirement. 

WSARA  203:  (a)  COMPETITIVE  PROTOTYPING.— Not  later  than  90 
days  after  the  date  of  the  enactment  of  this  Act,  the  Secretary  of 
Defense  shall  modify  the  guidance  of  the  Department  of  Defense 
relating  to  the  operation  of  the  acquisition  system  with  respect  to 
competitive  prototyping  for  major  defense  acquisition  programs. 

Prototyping  is  required  unless  the  requirement  is  waived  by  the  MDA, 
prototypes  exceed  the  expected  life-cycle  benefits,  or  the  DoD  is  unable  to  meet 
critical  national  security  objectives. 

We  assessed  that  prototyping  does  not  typically  happen  in  production 
phases.  Because  most  of  the  programs  at  PEO  CS&CSS  were  in  the  later  phases, 
the  need  to  prototype  was  not  as  prevalent.  However,  there  were  some  cost  savings 
incentives  to  prototype  if  modifications  to  systems  were  being  made.  In  that  case, 
the  PMs  typically  used  the  Research,  Development  and  Engineering  Center  (RDEC) 
to  develop  a  prototype  or  a  TDP  and  then  had  contractors  develop  their  solutions  for 
the  solicitation,  thus  improving  competition. 

In  contrast,  rapid  initiatives  (like  MRAP)  did  not  have  prototypes  as  per  the 
WSARA  requirement.  The  urgent  needs  of  the  Army  drove  the  requirement,  and 
they  had  companies  produce  units  based  on  performance  specs  to  get  equipment 
out  to  the  troops  faster  and  leverage  the  waiver  elements  of  the  legislation. 

4.  Systemic  Problems  and  Product  Termination 

For  this  section,  our  research  efforts  focused  on  the  policies  and  procedures 
employed  by  PEO  CS&CSS  concerning  identification  of  systemic  problems  and 
methods  of  determining  when  a  product  line  must  be  terminated  to  determine 
whether  deviations  from  WSARA  policy  intent  and  application  exist. 

The  systemic  problems  were  typically  identified  through  internal  reporting 
procedures  at  project  level  and  presented  to  the  MDA.  If  problems  are  identified, 
WSARA  Title  II  sec  204  requires  the  MDA  to  certify  costs,  needs,  program 
duplicates,  relevancy,  and  that  depot-level  maintenance  and  repair  capabilities  have 
been  made,  and  conduct  an  analysis  of  alternatives  consistent  with  DCAPE 
guidance. 
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WSARA  Section  206  addresses  product  termination.  The  PMs  reported 
changes  to  cost,  performance,  or  schedule.  If  the  changes  were  detrimental,  the 
WSARA  requires  the  PM  to  determine  the  root  cause  of  cost  growth  or  schedule 
threshold  breach.  The  reports  for  these  changes  followed  the  same  reporting 
processes  as  the  systemic  problems.  However,  it  is  the  MDA  that  ultimately 
determines  whether  a  product  or  project  is  continued  or  terminated. 

The  reporting  structure  in  PEO  CS&CSS  allowed  for  multiple  reports  on  cost 
performance  and  schedule  from  product  managers  up  to  the  project  managers  to  the 
PEO.  This  facilitated  the  requirement  from  WSARA  204  with  each  product/project 
manager  able  to  customize  their  reports  to  explain  an  issue  at  hand. 
Recommendations  made  at  the  PM  level  with  supporting  cost,  schedule,  and 
performance  data  facilitated  decision-making  capabilities  for  the  MDA  on  issues  and 
termination  if  needed. 

5.  Effectiveness  of  WSARA  Policies 

WSARA  policy  relevance  and  cost  effectiveness  for  PEO  CS&CSS  was 
addressed  to  determine  whether  there  is  added  value  from  this  policy  within  the 
PEO.  Cost  effectiveness  is  a  measure  of  the  results  added  by  a  system  or,  in  this 
case,  legislative  requirements  (Hagan,  2012,  p.  B-49).  We  determined  that  the 
WSARA  legislation  requiring  the  policies  that  PEO  CS&CSS  followed  were  generally 
cost  effective.  However,  there  was  some  distaste  for  the  reporting  requirements; 
although  deemed  necessary,  the  reports  may  not  provide  significant  value. 

F.  WSARA  IMPLEMENTATION 

For  this  section,  we  focused  our  inquiries  on  reporting  requirements  and 
personnel  evaluation  criteria  within  PEO  CS&CSS.  Studying  these  items  enhanced 
our  abilities  to  determine  what  is  important  to  the  PEO  and  whether  WSARA  policies 
are  among  the  list  of  important  items.  We  gauged  their  significance  by  level  and 
frequency  of  reporting  and  whether  WSARA  policy  items  are  included  in  personnel 
evaluation  criteria.  WSARA  assists  with  comparing  and  evaluating  the  importance  of 
these  reporting  policies  within  the  PEO  and  whether  they  appear  to  be  effective  and 
efficient. 

1.  Cost,  Schedule,  and  Performance  Reports 

The  GAO  study  mentioned  in  the  beginning  of  this  project  stated  that 
incentives  are  partially  to  blame  for  inadequacies  in  acquisitions  (Francis,  2009). 

We  wanted  to  determine  how  incentives  influenced  the  reporting  process.  We 
learned  that  acquisition  managers  and  leaders  are  incentivized  to  maximize  fund 
obligation  rates  more  than  anything  else.  Although  there  is  a  push  to  meet  the 
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needs  of  soldiers  and  that  is  typically  where  acquisitions  begin,  somewhere  along 
the  line,  it  becomes  about  obligating  funds. 

In  terms  of  what  is  reported,  the  research  indicated  a  wide  range  of  differences. 
Reform  efforts  (like  the  WSARA)  stimulate  additional  reporting.  Our  research 
indicated  that  there  are  more  reports  required  than  can  be  numbered.  The  acquisition 
process  becomes  muddied  when  reform  efforts  create  deviations  from  standard 
reporting  requirements.  Reporting  has  become  such  an  arduous  process  at  the  PEO 
that  personnel  must  be  allocated  solely  to  tracking  reports  and  ensuring  that  reports 
are  submitted  on  time.  Much  of  this  reporting  appears  to  be  required  from  those 
initiating  the  reform  efforts  so  that  these  individuals  have  evidence  that  their  reforms 
are  creating  change. 

Programs  at  PEO  CS&CSS  in  earlier  phases  of  the  acquisition  life  cycle  have 
fewer  reporting  requirements  than  those  in  the  latter  phases.  Some  of  the  common 
reports  include  a  weekly  report  sent  to  everyone  in  his  or  her  product  line,  weekly 
staff  calls,  metrics  and  portfolio  reviews,  and  Commander’s  Critical  Information 
Requirements  (CCIR)  reporting. 

We  determined  that  PEO  CS&CSS  management  trusts  that  all  of  its  reports 
go  to  a  higher  organization  beyond  its  PEO,  albeit  a  defense  acquisition  executive 
(DAE)  if  one  exists.  Army  Budget  Office,  or  Congress.  In  terms  of  which  reports  are 
more  important  than  others,  there  appears  to  be  a  focus  on  obligation  of  funds  and 
that  these  reports  are  prioritized  with  regard  to  what  is  submitted  to  higher  tiers. 

There  are  not  strict  reporting  structures.  We  determined  that  program  cost 
reporting  is  stressed  above  all  else  when  we  studied  PEO  reporting  hierarchies. 
Especially  in  times  of  fiscal  constraints,  reporting  has  become  even  more  about  cost. 
However,  this  is  not  reporting  to  show  how  costs  are  mitigated.  Rather,  this 
reporting  focuses  on  obligation  rates.  There  seems  to  be  little  concern  with  holding 
managers  to  items  mentioned  within  WSARA  Title  II  policies.  The  focus  is  on 
ensuring  funds  are  obligated  fully  and  on  time.  None  mentioned  having  to  report 
specifically  on  any  aspect  of  the  WSARA  reforms. 

2.  Reporting  Management 

Report  oversight  is  another  factor  used  in  our  research  to  indicate  the 
importance  of  what  is  reported.  Those  items  with  greatest  oversight  are  determined 
to  be  most  important  to  the  PEO  and  their  superiors.  Our  analysis  for  this  section 
focused  on  PEO  CS&CSS  reporting  processes,  including  the  number  of  reports,  and 
the  level  of  oversight  or  senior  management  involvement.  The  level  of  oversight 
varied  based  on  where  a  respective  program  fell  within  the  acquisition  life  cycle. 
Those  programs  still  in  the  early  phases  of  the  acquisition  life  cycle  stated  that  they 
feel  there  is  little  oversight  to  what  they  do  on  a  regular  basis.  However,  managers 
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of  programs  operating  in  the  latter  phases  of  the  acquisition  life  cycle  stated  they  are 
“over  sighted  to  death.” 

Although  oversight  varied  depending  on  acquisition  life-cycle  phase,  there 
was  a  consensus  that  reports  were  not  actually  reviewed  beyond  the  PEO  level 
based  on  the  absence  of  feedback.  Often,  feedback  is  received  concerning 
reporting  only  if  there  is  a  combat-related  requirement.  For  instance,  if  soldiers  are 
wounded  or  killed  in  combat  while  employing  a  product  in  their  program,  this 
generates  changes  to  an  aspect  of  their  product  (armor  for  example),  and  oversight 
becomes  abundant.  Otherwise,  for  programs  in  early  acquisition  phases,  there 
seems  to  be  little  oversight  as  to  what  is  going  on  in  a  program.  When  a  problem 
presents  itself  in  a  program  is  another  instance  of  when  report  feedback  is 
prominent.  Specifically,  reports  concerning  funding  problems  receive  an  abundance 
of  feedback  along  with  additional  oversight  indicating  that  funding  and  cost  are 
valued  above  all  other  report  items. 

3.  Performance  Assessments  and  the  WSARA 

Performance  evaluations  and  metrics  are  a  method  employed  by  supervisors 
to  incentivize  behavior  in  congruence  with  organizational  goals.  Investigating 
whether  personnel  evaluations  within  PEO  CS&CSS  included  requirements  from  the 
WSARA  demonstrates  the  level  of  the  WSARA  policies  significance  to  the  PEO.  We 
focused  our  research  on  specific  WSARA  aspects  including  reporting,  cost  control, 
and  requirements  development,  and  analyzed  whether  these  policies  were 
evaluated  in  either  the  civilian  or  military  performance  evaluations.  We  discovered 
that  personnel  are  not  evaluated  on  (nor  evaluate  their  subordinates  on)  anything 
directly  stemming  from  WSARA  policies.  Considering  adherence  to  policy  initiatives 
when  conducting  evaluations  is  something  worth  implementing  for  managers  and 
those  under  their  evaluation.  Evaluations  at  PEO  CS&CSS  are  more  about 
percentage  of  funds  obligated  or  number  of  upgrades  installed,  for  example,  than 
they  are  about  managing  cost,  schedule,  and  performance. 

There  is  discontinuity  between  civilian  and  military  personnel  evaluations  as 
well,  according  to  those  interviewed.  Civilian  evaluations  are  very  vague  and  do  not 
tell  much  about  their  performance  unless  they  did  something  wrong.  The  evaluation 
process  is  further  complicated  due  to  lacking  longevity  of  military  personnel  within  a 
program  office.  As  such,  evaluations  become  more  about  what  kind  of  person  the 
military  individual  is,  rather  than  how  they  manage  a  program  with  regard  to  items 
such  as  those  mentioned  in  the  WSARA  policies.  Because  that  individual  is  not  in 
the  program  from  conception  through  final  phases  of  product  acquisition,  it  is  difficult 
to  measure  their  cost,  performance,  and  schedule  contributions. 
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4.  Reporting  Effectiveness 

We  assessed  the  effectiveness  of  the  PEO  CS&CSS  reporting  process  and 
contents  of  submitted  reports  to  identify  whether  inefficiencies  exist  and  to  ascertain 
the  implications  of  continued  acquisition  reforms  (such  as  the  WSARA)  on  this 
process.  We  established  that  the  current  reporting  process  is  not  effective.  The 
consensus  was  that  the  acquisition  process  itself  is  very  effective.  However, 
everything  that  is  continually  added  on  top  of  this  process,  such  as  WSARA  reforms 
and  the  reporting  requirements  that  come  with  reform,  take  away  from  the  process’s 
effectiveness.  There  are  opportunity  costs  associated  with  having  to  obligate 
personnel  in  a  program  to  managing  nothing  but  reporting  requirements.  This  takes 
away  from  their  abilities  to  add  real  value  to  other  aspects  within  a  program. 

Additionally,  there  is  little  feedback  with  regard  to  the  reports  that  they  submit 
on  a  regular  basis.  New  policies  often  create  new  reporting  requirements,  but 
acquisition  managers  are  not  confident  that  these  reports  amount  to  much  more  than 
simply  satisfying  a  requirement  to  report.  Policy  reforms  that  lead  to  additional 
reporting  requirements  appear  to  be  nothing  more  than  another  person’s  (the  person 
implementing  the  reform)  attempt  to  have  their  part  in  history.  This  leads  to 
ineffective  reporting  due  to  the  time  it  takes  to  generate  these  reports  and  the  tasks 
that  may  fall  by  the  wayside  because  time  was  spent  reporting  rather  than  managing 
cost,  schedule,  performance  and  risk. 

Initial  reporting  requirements  incumbent  to  the  baseline  acquisition  process 
are  adequate.  An  acquisition  reform  such  as  the  WSARA  adds  to  the  reporting 
requirements.  As  more  reform  initiatives  are  made,  more  reporting  is  required. 
Further,  no  unnecessary  or  outdated  reporting  requirements  are  eliminated.  These 
new  and  old  reporting  requirements  compound  on  top  of  each  other  until  only 
reporting  is  happening.  The  original  intent  of  the  reporting  is  lost,  and  no  one  has 
any  idea  why  the  reporting  is  being  done  and  whether  reports  are  necessary  at 
higher  levels  anymore. 

G.  POLICY  IMPLEMENTATION  AND  GAPS 

This  section  presents  information  demonstrating  whether  there  was  deviation 
from  the  WSARA’s  intent  and  application  within  PEO  CS&CSS,  and  if  so,  at  what 
level,  and  what  is  actually  implemented  within  programs. 

1.  Tracking  Acquisition  Policy 

Acquisition  policy  and  reform  is  constantly  being  reviewed  and  updated.  This 
process  is  managed  through  policies  at  all  levels  of  the  bureaucracy.  Acquisition 
managers  within  PEO  CS&CSS  have  challenges  tracking  changes  to  policy.  Some 
policy  changes  or  updates  to  procedure  are  relatively  simple  to  print  and  make  a 
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cheat  sheet  to  check  program  efforts  against.  However,  complex  policy  reforms  and 
legislative  changes  such  as  the  WSARA  are  more  difficult,  and  this  difficulty  is 
compounded  when  one  considers  that  the  WSARA  is  only  one  of  many  policy 
documents  requiring  adherence. 

Additionally,  PMs  are  not  pressured  to  continually  check  their  programs 
against  policy.  Resources  such  as  the  Defense  Acquisition  Portal  (DAP)  and 
Defense  Acquisition  University  (DAU)  were  cited  as  potential  sources  to  consult 
when  searching  for  policy  documents  when  evaluating  programs.  But  given  the 
amount  of  policy  and  reforms  that  exist,  this  can  be  a  lengthy  process. 

True  policy  reviews  come  when  the  program  is  reaching  certain  milestone  or 
decision  briefings  within  the  acquisition  process.  At  these  points,  the  policymaking 
personnel  scrub  documents  and  procedures  against  policy  requirements,  which 
assists  with  ensuring  the  program  is  in  compliance.  However,  due  to  the  ever- 
increasing  amount  of  change  and  reforms  that  takes  place  with  the  acquisition 
process,  the  time  it  takes  to  process  these  documents  and  briefings  is  also  ever 
increasing. 

2.  Policy  Challenges 

The  requirements  created  by  the  WSARA  created  bureaucratic  challenges. 
One  of  the  programs  mentioned  having  difficulties  implementing  competition 
throughout  the  life  cycle  of  their  program  due  to  TDP  issues.  The  program 
acknowledges  that  full  and  open  competition  is  a  requirement;  however,  if  one 
contractor  holds  the  TDP  for  a  piece  of  equipment  or  system,  competition  is 
compromised.  Purchase  of  the  TDPs  from  the  contractor  is  often  impossible  in 
circumstances  where  the  contractor  put  the  TDPs  out  of  reach  from  the  government 
by  placing  such  a  high  cost  on  the  TDPs  that  it  is  outside  the  program’s  funding 
capabilities. 

Policy  reforms  and  changes  are  difficult  to  implement  based  on  how 
cumbersome  the  process  becomes.  When  a  policy  is  reformed,  it  must  be 
implemented.  Then  personnel  must  be  employed  to  check  policies  and  generate  a 
database  for  ensuring  a  program  is  evaluated  against  the  policy  efforts.  Additional 
personnel  are  then  deviated  from  their  prior  duties  to  start  reporting  and  ensuring 
that  the  database  and  reporting  is  feeding  the  individuals  who  require  the  reporting. 
Frustration  with  this  process  abounds  because  it  shows  that  reporting  compliance 
with  policy  is  more  significant  than  actually  ensuring  correct  policy  implementation 
and  sound  program  management. 

The  major  roadblock  programs  have  encountered  with  implementing  the 
competition  policy  from  the  WSARA  is  the  TDP  issue.  Also,  the  cumbersome 
process  of  tracking,  creating  metrics  to  measure  a  program  against,  reporting,  and 
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tracking  of  reporting  requirements  that  are  created  with  each  additional  change  to 
the  acquisition  process  are  disincentives  to  policy  implementation. 

3.  Policy  Successes 

The  requirements  created  by  the  WSARA  have  created  opportunity  for 
success.  Being  forced  to  consider  trade-offs  between  cost,  schedule,  and 
performance  in  the  early  years  of  a  program  changes  acquisition  approaches. 
Successes  have  occurred  through  utilization  of  the  DoDI  5000  process  guides,  and 
other  instances  persist  where  individuals  within  their  programs  utilized  various 
policies  to  produce  rapid  results  when  a  combat  need  was  identified. 

Positive  feedback  in  programs  was  not  directly  linked  to  WSARA  policy.  Most 
positive  things  that  happen  in  programs  result  from  following  acquisition  policy  and 
endeavoring  to  meet  the  needs  of  end  users.  However,  the  correlation  between 
these  positive  outcomes  and  implementation  of  a  WSARA  Title  II  policy  is  not  easily 
delineated. 
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V.  CONCLUSIONS  AND  RECOMMENDATIONS 


A.  INTRODUCTION 

The  final  section  of  this  project  addresses  our  research  conclusions  and 
recommendations.  These  items  are  grouped  within  the  categories  of  quantitative, 
organizational  and  procedural,  Title  II  WSARA,  and  summary  conclusions  and 
recommendations.  We  initially  address  statistical  conclusions  drawn  from  the 
DAMIR  analysis.  The  remaining  conclusions  and  recommendations  are  based 
primarily  on  interview  trends,  as  well  as  any  other  research  findings. 

1.  Conclusions  Summary 

There  is  no  direct,  measurable  linkage  between  Title  II  WSARA  policies  and 
improvements  to  processes,  organizational  structure,  costs,  schedule,  and 
performance.  Priorities  and  incentives  drive  the  acquisition  process.  Within  the 
acquisition  process  and  as  new  acquisition  officers,  we  are  constantly  counseled  to 
consider  cost,  schedule,  and  performance  within  acquisition  programs.  Indeed,  the 
research  indicated  that  these  three  items,  along  with  risk,  are  equally  important. 
However,  as  we  conducted  interviews,  discrepancies  surfaced  regarding  how  these 
items  are  prioritized  within  a  program. 

Managers  and  workers  of  programs  respond  to  incentives.  Incentives  present 
themselves  in  the  form  of  career  advancement,  excellent  evaluations,  praise,  and 
many  others.  When  and  why  these  incentives  are  delivered  to  a  recipient  establish 
the  program’s  priorities.  Therefore,  while  a  manager  may  say  cost,  schedule, 
performance,  and  risk  are  equally  important,  his  or  her  actions  in  rewarding  only 
when  funds  are  fully  obligated  lead  subordinates  to  determine  that  the  true  priority  is 
money  obligated  and  spent  in  a  program  rather  than  the  stated  priorities. 

Adding  to  the  already  numerous  policy  documents  that  exist  to  govern  the 
acquisition  process  is  not  the  most  effective  or  efficient  manner  to  improve 
acquisitions.  Rather,  enforcing  requirements  for  PdMs  and  other  representatives  to 
adhere  to  the  acquisition  process  while  establishing  priorities  at  the  PEO  level  and 
below  (priorities  determined  from  evolving  needs  and  fiscal  requirements)  and 
incentivizing  all  involved  to  adhere  to  these  priorities  will  deliver  desired 
improvements. 

B.  CONCLUSIONS 

1 .  Quantitative 

It  is  clear  that  MDAP  cost  variability  is  in  more  control  on  current  estimate  and 
PAUC  and  APUC  baseline  comparisons  since  the  implementation  of  the  WSARA. 
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As  stated  in  the  data  analysis  section,  we  focused  on  analyzing  the  measures 
of  central  tendency  for  the  selected  elements  of  cost  in  the  current  estimate,  PAUC 
change  against  baseline,  and  APUC  change  against  baseline  from  each  Army 
MDAP  SAR  that  was  available  from  2005  to  201 2. 

The  reader’s  focus  should  shift  to  the  coefficient  of  variation  (CV),  because 
this  measurement  equalizes  the  playing  field  because  it  is  has  no  unit,  nullifying 
actual  dollar  figures  or  program  baseline  estimates  and  comparing  the  program 
standard  deviation  divided  by  its  average,  giving  relative  magnitude  and  relative 
variability.  For  example,  from  Jeff  Sauro’s  (201 2)  website, 

if  the  mean  is  80  and  standard  deviation  is  1 2,  the  cv  =  1 2/80  =  .1 5  or 
1 5%.  If  the  standard  deviation  is  .20  and  the  mean  is  .50,  then  the  cv 
=  .20/.50  =  .4  or  40%.  So  knowing  nothing  else  about  the  data,  the  CV 
helps  us  see  that  even  a  lower  standard  deviation  doesn’t  mean  less 
variable  data. 

We  calculated  the  differences  of  CV  between  the  samples  (for  each  SAR  cost 
element)  before  the  WSARA  and  then  after  WSARA  implementation. 

For  our  example,  reference  Table  1  of  this  project.  All  of  the  MDAPs’  APUC 
CV  before  the  WSARA  was  99.01%  and  after  was  81 .01%,  showing  the  change  of 
-1 8.78%  in  Table  2.  For  the  programs  that  only  decreased  in  CV,  they  did  so  by 
-85.34%,  as  shown  in  Table  2  (with  the  data  computation  drawn  from  Table  1). 

Each  of  the  values  in  Table  2  that  decreased  indicates  a  cost  measurement  that  is 
more  in  control  (and  less  in  control  if  the  CV  increased). 

Since  WSARA  implementation,  the  sampled  programs  have  improved  in  cost 
control.  The  spirit  of  the  law  intended  to  have  more  control  on  programs  and  lower 
costs. 

Table  2  summarizes  the  changes  on  programs  that  increased  or  decreased 
before  the  implementation  of  the  WSARA  and  after.  In  addition,  there  is  a  combined 
net  change  in  each  measurement  showing  the  total  change. 
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Table  2.  Summary  SAR  MDAP  Changes  in  Central  Tendency 


PROGRAMS  THAT  INCREASED 


SAR  Report  Data_ Average_ Stdev_ Cy 


Current  Estimate  $160.09  $900.64  13.64% 

PAUC%  Change  34.88%  0.1803  58.82% 

APUC%  Change  6.10%  0.0671  33.40% 


PROGRAMS  THAT  DECREASED 


SAR  Report  Data_ Average_ Stdev 


Current  Estimate  $(4,538.13)  $(970.56)  -14.24% 

PAUC%  Change  -7.45%  -0.0596  -131.12% 

APUC%  Change  -9.15%  -0.0655  -85.34% 


COMBINED  NET  CHANGE  FOR  ALL  PROGRAMS 


SAR  Report  Data 

Average 

Stdev 

Cl^ 

Current  Estimate 

$(1,514.21) 

$(306.97) 

-2.27% 

PAUC  %  Change 

4.98% 

0.0379 

-69.63% 

APUC  %  Change 

-4.06% 

0.0125 

-18.78% 

It  is  clear  that  MDAP  cost  variability  is  in  more  control  on  current  estimate  and 
PAUC  and  APUC  baseline  comparisons  since  the  implementation  of  the  WSARA. 

2.  Organizational  and  Procedural  Impacts 

Our  research  indicated  no  true  change  to  organizational  structure,  task 
allocation,  or  administrative  functions  directly  linked  to  WSARA  implementation.  The 
organizational  structure,  at  the  PEO  level  and  above,  is  very  structured.  This  is 
necessary  to  establish  clear  linkages  and  lines  of  responsibility  from  PEO  to 
congressional  leaders,  DAEs,  and  others.  However,  at  the  PdM  level  and  below,  the 
organizational  structure  is  highly  fluid  and  was  different  within  each  program  that  we 
interviewed,  as  were  their  procedures.  The  structure  was  tailored  to  focus  personnel 
on  the  priorities  of  the  program.  Some  PdMs  managed  people  and  tasks  by 
functional  category  (logistics,  operations,  administration,  engineering,  etc.)  while 
others  were  organized  by  product  line  or  subcategory  within  the  program  itself. 

This  decentralized  approach  to  managing  structure  and  procedures  at  the 
PdM  level  and  below  is  effective  and  efficient  according  to  program  leaders. 

Although  none  interviewed  stated  the  WSARA  as  a  reason  for  changing 
organizational  structure  and  procedures,  it  is  possible  that,  due  to  influence  from 
representatives  at  the  PEO  level  and  above,  whose  actions  may  have  been 
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motivated  by  the  WSARA,  the  reform  is  partially  responsible  for  some  organizational 
changes. 

The  research  suggests  that  a  negative  organizational  and  procedural  impact 
resulting  from  acquisition  reforms  and  changes  to  policy.  This  negative  impact  is 
seen  in  the  increased  amount  of  reporting  and  tracking  or  reporting  that  is  required 
each  time  new  policy  is  implemented.  Increased  reporting  leads  to  allocation  of 
resources  (personnel  and  time)  to  reporting  requirements,  decreasing  resources  that 
have  previously  been  focused  on  other  important  management  tasks. 

3.  WSARA  Title  II  Policy  Application  Conclusions 

Title  II  WSARA  policies  focus  on  evaluation  of  tradeoffs  between  cost, 
performance  and  schedule,  prototyping,  maintaining  competition  throughout  a 
program’s  life  cycle,  addressing  systemic  problems,  and  terminating  product  lines. 
Throughout  our  initial  research  and  program  representative  interviews,  we 
determined  that  the  acquisition  process  and  policies  already  have  procedures  and 
rules  concerning  what  Title  II  WSARA  requires. 

Additionally,  not  all  programs  fall  in  the  same  space  and  time  within  the 
acquisition  process.  Therefore,  when  a  new  requirement  to  ensure  competition 
throughout  a  product’s  life  cycle  is  implemented  in  a  program  in  its  latter  (Operations 
and  Support)  phase  of  the  acquisition  process,  this  may  lead  to  schedule  delays. 

The  research  indicated  that,  in  such  programs,  the  TDPs  are  often  out  of  their  reach. 
New  competition  requirements  likely  require  waivers  if  competition  cannot  take 
place,  which  may  result  in  schedule  overruns  due  to  processing  delays.  This  is  one 
example  of  many  possible  scenarios  where  policy  changes  lead  to  delays  or  other 
tradeoffs  within  programs. 

C.  RECOMMENDATIONS 

The  quantitative  findings  indicate  programs  that  are  lower  and  more  in  control 
on  average  since  the  implementation  of  WSARA.  However,  a  larger  sample  size  of 
cost  data  from  SARs  or  other  sources  would  increase  statistical  significance. 
Although  there  is  no  direct  causation  identified,  there  is  a  clear  correlation  of  the 
WSARA’s  impacts  on  Army  MDAPs.  We  recommend  continuing  to  implement 
WSARA  policy  requirements  and  lessening  other  acquisition  reform  to  control  for  the 
WSARA  cause  and  effect. 

To  say  that  each  program  is  the  same  and  thereby  should  maintain  a  pre¬ 
designated  organizational  structure  at  the  PdM  level  and  below  would  force 
programs  to  reorient  themselves  in  a  manner  that  is  likely  to  be  less  capable  and 
less  effective  than  they  currently  are.  PdMs  must  continually  evaluate  their 
organizational  structure  and  procedures  in  order  to  determine  how  to  best  tailor  their 
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programs  to  meet  the  requirements  in  accordance  with  where  their  program  falls  in 
the  acquisition  process  and  priorities  established  by  their  PEO  and  above. 

PEO  and  above  representatives  should  be  responsible  for  establishing 
procedural  and  organizational  priorities  that  are  in  line  with  policies  and  reforms. 
People  respond  to  incentives  and  react  according  to  that  which  is  rewarded.  When 
PEOs  establish  priorities  and  evaluate  based  on  these  priorities,  it  ensures  PdMs 
focus  on  those  priorities  without  needing  to  know  each  change  to  policy  that  occurs. 

Our  research  found  that  policymakers  may  not  be  evaluating  whether  a  policy 
reform  is  required  prior  to  implementing  a  change.  In  other  words,  policy  changes 
become  more  about  a  policymaker  wanting  to  make  his  or  her  mark  on  history  while 
responding  to  pressures  from  external  forces,  such  as  public  criticism  to  fix  military 
acquisitions.  Thus,  a  change  becomes  implemented  that  requires  reporting, 
waivers,  and  tracking,  or  all  of  these  items,  resulting  in  a  potential  detriment  to 
effective  program  management. 

Policymakers  must  evaluate  the  baseline  acquisition  process  and  its 
supporting  policy  by  ensuring  that  reforms  are  not  made  when  they  are  not  needed. 
Further,  reforms  need  not  be  repetitive  in  nature.  When  repetitive  changes  are 
made  to  policies,  this  is  an  indicator  that  incentives  and  priorities  are  mismatched.  A 
new  reform  is  not  necessary.  Rather,  PEOs  and  PdMs  need  to  understand  what 
their  priorities  are  and  be  managed  through  incentives  to  adhere  to  those  priorities. 
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